A business incurs much higher charges when attempting to win new customers than to retain existing ones. As a result, much research has been invested into new ways of identifying those customers who have a high risk of churning. However, customer retention efforts have also been costing organizations large amounts of resources.
A business incurs much higher charges when attempting to win new customers than to retain existing ones. As a result, much research has been invested into new ways of identifying those customers who have a high risk of churning. However, customer retention efforts have also been costing organizations large amounts of resources.
Original Title
Applying Data Mining to Customer Churn Prediction in an Internet Service Provider
A business incurs much higher charges when attempting to win new customers than to retain existing ones. As a result, much research has been invested into new ways of identifying those customers who have a high risk of churning. However, customer retention efforts have also been costing organizations large amounts of resources.
A business incurs much higher charges when attempting to win new customers than to retain existing ones. As a result, much research has been invested into new ways of identifying those customers who have a high risk of churning. However, customer retention efforts have also been costing organizations large amounts of resources.
International Journal of Computer Applications (0975 8887)
Volume 9 No.7, November 2010
8 Applying Data Mining to Customer Churn Prediction in an Internet Service Provider
Afaq Alam Khan BGSB University J&K, India Sanjay Jamwal BGSB University J&K, India M.M.Sepehri Tarbiat Modares University, Tehran, Iran.
ABSTRACT A business incurs much higher charges when attempting to win new customers than to retain existing ones. As a result, much research has been invested into new ways of identifying those customers who have a high risk of churning. However, customer retention efforts have also been costing organizations large amounts of resources. Same is the situation in ISP industry in I.R.Iran. Exploiting the use of demographic, billing and usage data, this study tends to identify the best churn predictors on the one hand and evaluates the accuracy of different data mining techniques on the other. Clustering users as per their usage features and incorporating that cluster membership information in classification models is another aspect which has been addressed in this study. General Terms Data Mining & Churn Keywords ISP, Churn, Data mining, Decision tree, Regression, Neural network. 1. INTRODUCTION Internet users worldwide are increasing at impressive rate. According to the statistics published by Miniwatts Marketing Group there are 1,043,104,885 internet users in the world with 8.4 million unique websites. Internet Service providers (ISP) the companies that sell internet access in various packages, are trying hard to add more and more customers to their system as this helps them to keep the price of service low and thus results in attracting further more customers [9]. Offering a variety of services to the customers is yet another way of ISPs to increase customer base. Customers on the other side come face to face with a decision of choosing an ISP. They become more demanding and tend to switch from previous service provider to another service provider. This gives birth to the notion of churn. Churn or customer attrition is defined as the annual turnover of the market base [23]. Keeping in view that cost of obtaining new customer is five times higher than maintaining an existing customer [8] and ISPs spent a huge amount on advertisement to gain a customer, ISPs do not afford to lose their existing customers and need to fight customer churn. There are two basic approaches to fight customer churn. Untargeted approaches rely on superior product and mass advertising to increase brand loyalty and retain customers. Targeted approaches rely on identifying customers who are likely to churn, and then either provide them with a direct incentive or customize a service plan to stay. There are two types of targeted approaches: reactive and proactive. Adopting a reactive approach, a company waits until customers contact the company to cancel their (service) relationship. The company then offers the customer an incentive, for example a rebate, to stay. Adopting a proactive approach, a company tries to identify customers who are likely to churn at some later date in advance. The company then targets these customers with special programmes or incentives to keep the customers from churning. Targeted proactive programs have potential advantages of having lower incentive costs and because customers are not trained to negotiate for better deals under the threat of churning. However these systems can be very wasteful if churn predictions are inaccurate, because then companies are wasting incentive money on customers who would have stayed anyway. That is why the customer churn prediction process need to be as accurate as possible [6]. 2. REVIEW OF LITERATURE Customer attrition is an important issue for any company and is easiest to define in subscription based businesses, and partly for that reason, churn modeling is most popular in theses businesses [5]. Long-distance companies, Mobile phone service providers, Insurance companies, Cable companies (Pay-TV) [6], financial service companies, Internet service providers, newspapers, magazines, and some retailers all share subscription model where customers have a formal, contractual relationship which must be explicitly ended. 2-1. Customer Churn in Mobile service providers: As deregulation, new technologies, and new competition have opened up telecommunication industry, the telecommunication service market has become more competitive than ever. And in this strongly competitive and broadly liberalized mobile telecommunication industry, customer churn has turned into very serious issue. Many subscribers frequently churn from one provider to another in search of better rates/service or for the benefits of signing up with new carrier (e.g., such as receiving the latest cellular phone). Low signup fees particularly for prepaid mobile services also encourage customers to churn. Wireless local number portability (WLNP) or the ability to change mobile carriers and keep mobile phone number also poses a big challenge to the already reeling telecommunication company. It is estimated that the average churn rate for the mobile telecommunications is 2.2% per month. i.e., about 27% of given carriers subscriber are lost each year [31], making it International Journal of Computer Applications (0975 8887) Volume 9 No.7, November 2010 9 essential to develop an effective churn reduction method. The cost of acquisition of new mobile service subscriber is estimated to be from $300 to $ 600. However, the cost of retaining an existing subscriber is generally much lower than that [30]. Wei, et al [30] used the call pattern changes and contractual data and developed a data mining based churn prediction technique that identify potential churners at the contract level. In their study they used the decision tree approach as the basis for the development of their technique for mobile telecommunication company in Taiwan. In an another attempt to identify the determinants of subscriber churn and customer loyalty in Korean mobile telephony market; Kim et al [17] used binomial logit model based on the survey of 973 mobile users . 2-2. Customer churn in banking and insurance: Over the past two decades, the financial markets have become more competitive due to the mature nature of the sector on the one hand and deregulation on the other, resulting in diminishing profit margins and blurring distinctions between banks, insurers and brokerage firms. Hence, now a day a small number of large institutions offering a wide set of services dominate the financial- service industry. These developments stimulated bank and Insurance companies to implement customer relationship management (CRM). Under this intensive competitive pressure, companies realize the importance of retaining, their current customers. The substantive relevance of attrition modeling comes from the fact that a bank is able to increase its profits by 85% due to a 5% improvement in the retention rate [26]. Bart et al [32] studied the defectors of saving and investment (SI) customers of a large Belgian financial service provider. Their study involves the use of Kplan-Meier estimator to gain insight into the timing of the SI churn event. A multi dimensional probit model and a proportional hazard model are performed to find the most convenient products to cross sell in terms of customer preferences and the likelihood to lower the customer defection proness respectively. In order to establish a system that can provide warning signs to network banking services soon before they are losing valuable customers, Ding-An Chaing et. al. [35] took association rules in use and proposed an efficient algorithm called goal- oriented sequential pattern, which can find out behavior patterns of loosing customers or clues before they stop using some products. 2-3. Customer churn in ISP: Internet popularity is growing at impressive rate. Sooner or later, every customer comes face to face with a decision of choosing an ISP. ISPs endure a five times higher churn rate cumulating to 10% monthly [23]. It is also said that nearly half of all Internet subscribers leave their provider every year [2]. IP network metering, which has lately pick up steam, is the key enabler of getting the fundamental information of churn prediction. Shen- Tun Li et. al. [33] applied the well known data mining methodology CRISP-DM to investigate network usage behaviors of the ISP subscribers in Taiwan. They used Attribute-oriented Induction (AOI) method for discovering characteristics and discrimination knowledge of ISP customers from the ISP traffic data. An empirically tested model developed by Jyh-Shen Chiou [34] examines the antecedents of consumer loyalty toward ISPs. In choosing an ISP, pricing was and is a main differentiating factor in mind of customers; however, service quality has emerged as a major concern among users lately. Management of ISP has discovered that service quality is important not only for attracting new customers, but more importantly for retaining existing customers who may otherwise be lured away by lower fees. [20] developed a probability model for ISP subscriber churn. Their model relates the probability of subscribers churn to various service attributes and subscriber characteristics. 3. ISP Industry in Iran In recent years, government of I.R.Iran has taken extensive and vigorous actions to increase access to Internet. One of the objectives of the next five year development plan is to raise internet penetration rate from current 9.74% to 30% state wide. To meet the above mentioned objective, four layer architecture shown in figure 1, has been deployed by government to provide different internet services to different kinds of users.
(Figure 1: Four layer Architecture of internet services in I.R.Iran)
ICP(Internet Connection Providers): The first tier services are provided by ICPs. The role of ICP is to buy Internet connections in large volumes from international vendors. They distribute the capacity in large chunks to ISPs and volume consumers. The second tier of Internet Services includes three groups of providers which are as follows: PAP(Public Access Provider): The group via DSL links provides High speed access for their customers. IDC(Internet Data Centre): Provides hosting services with creating secure storage spaces and offering software services on the internet network. ISP(Internet service Provider): provide opportunity for inexpensive access to internet via telephone lines (WSIS, 2005).
Table 1: Number of issued permits VOIP IDC PAP ICP ISP 59 3 13 31 658
International Journal of Computer Applications (0975 8887) Volume 9 No.7, November 2010 10 Most of Internet users in Iran use dial-up internet connection. Users connect to their service provider through their phone line and need to authenticate before they can use any service. Internet service providers on the other hand distribute the authentication cards in the market. Each ISP has different types of cards with different prices. As there are more than 50 ISPs alone in Tehran, the capital city of Iran (www.irantelecom.ir), these ISPs have filled the market with different types of authentication cards and offer various discounts and incentives every now and then, and thus encourage the users to switch from one service provider to another. Taking in consideration the large number of ISPs, and more or less the saturation in this industry, ISPs need to shift their focus from customer accusation to the retention of customers, in particular the churn prediction and for this ISPs need to know their customers and their purchase history and usage pattern. But unfortunately most of the ISPs in Iran have no profile of their dial-up users and does not log their purchases and usage. In this way every time user buys a new authentication card and connects to the ISP, he is treated as a new customer. Sepanta Co. is one of the ISPs who operate in Tehran and has solved the above mentioned problem by encouraging its users to register all the authentication cards they purchase in their user area so that they can participate in the lucky draw every night at 00:00 hours. Sepanta Co. thus has the profile of its dial-up users including their purchases at different times and above all log the usage of each customer. Like all other service providers Sepanta Co. also suffers from the customer churn. Average monthly churn of 22% was estimated for this ISP. Applying KDD methodology, this study exploit the use of data provided by Sepanta Co. and presents a churn prediction model to meet the objectives mentioned below: a) Detection of as many churners as possible. b) To determine the best predictors among demography, billing and usage features. c) To compare different techniques of classification. Although the Said ISP offers different services including dial-up, DSL, VOIP etc, but we used dial-up users only in this study. First of all Dial-up being the prepaid service is ideal for performing churn prediction, secondly there is a fierce competition for this service and finally no usage data was available for VOIP customers. 4. METHODOLOGY KDD is defined as the nontrivial process of identifying valid, noval, potentially useful and ultimately understandable patterns in data. Based on the kind of the knowledge which can be discovered in databases, KDD techniques can be broadly classified in to several categories, including classification, clustering, Association, estimation, etc. In view of the fact that we deal with the discrete valued target variable, and our ultimate goal is to declare each customer potentially churner or potentially nonchurner, so the KDD function of our problem is defined to be a classification problem [10]. Following a typical KDD framework [13], where data mining is the core in the overall process, the experiment went through all the steps mentioned below.
4.1 Data works Sepanta Co. put three of its main tables at our disposal. tblSubscriber, tblCards and tblUsage are the three tables containing 65000, 400000 and 15000000 records respectively. As it is completely optional for users to register their cards and participate in lucky draw, so large number of rows were found in tblUsage whose related information was not available in other two tables and were of no use for our work. tblUsage table contain the data from 21/3/2005 to 21/5/2006 while there were subscribers who belong to the date before 21/3/2005 also present in the tblSubscriber table, which were again of no use for our analysis because their usage data was not present. Similarly declaring churn status of the subscribers who registered after 21/11/2005 was not possible. So as a first step, all those records from the three tables were deleted which protect us to define relationship between these tables. In this way tblSubscriber, tblCards and tblUsage were reduced to 9148, 61993 and 2093120 records respectively. Customers whose registration date lies with in a same month of Persian calendar were selected from tblSubscriber table and their purchase were searched in tblCards table after a certain date (i,e after six months from the date of their registration). If customer still had a purchase he was declared as nonchurner else churner.
Figure 2: Declare churn status As shown in figure 2 all those customers whose registration date lies between 21/3/2005 and 21/4/2005 were selected from tblSubscriber table and it was searched in tblCards table whether they have any purchases after 21/09/2005. Those who still had a purchase were declared nonchurners and rest of them were churners. Same was done with the subscribers who registered in the next seven months. Only 8 months from the tblSubscribers was taken in use because it was not possible to set the churn status for the rest of the subscribers. That is why the subscribers who registered after 21/11/2005 were deleted from the system including their cards and usage data.
4.2. Feature extraction Features which have been used for churn prediction in most of the related works can be categorised in to demographic, billing International Journal of Computer Applications (0975 8887) Volume 9 No.7, November 2010 11 and usage features [10][20][30]. Besides the straight forward demographic and billing features extracted from the given attributes, some usage features were constructed which are worth to explain here. Whole life time of each subscriber was divided into subperiods (15 days each) and total number of minutes (MOU) a subscriber connected to the system during each subperiod was calculated. Only 12 latest subperiods were taken into consideration. It will not be inappropriate to mention here that more recent usage behavior would be more useful for churn prediction [30]. Frequency of use (FOU) i.e. the number of times a subscriber connected to the system during each subperiod and the Volume of data (VOD) transferred during each subperiod were also calculated. Instead of adding these 36 usage attributes to the dataset, k-means clustering technique was taken in use and all the subscribers were clustered three times as per MOU, FOU and VOD. In this way we add three features to our dataset which represent the usage behaviour of the subscriber. Incorporating the cluster membership information to the classification model has shown better performance in the previous work [15][26]. Number of clusters chosen for all the three times was 7 as it showed the highest chi-square value. Portion of use (POU) is another feature related to the different portions of the day. Usage timing for different users is different. Some are day users and some others connect to internet at nights. Total usage of a subscriber was divided into six different portions (Table 2) to capture the usage behavior of subscribers.
Table 2: Six Usage Portions of the day Day Portion Time From To Evening 18:00 21:59 Latenight 22:00 01:59 Midnight 02:00 05:59 Morning 06:00 09:59 Midday 10:00 13:59 Afternoon 14:00 17:59
All the subscribers were once again clustered as per six attributes of POU and the cluster membership information was once again incorporated to the classification model. 4.3. Modelling Microsoft Business intelligence development studio was used to perform the task of classification. An object called Percentage Sampling shown in figure 3 let us divide the dataset into training and test sample.
Figure 3: Data flow diagram in MS Business Intelligence Development Studio
As was recommended in the previous work, we assigned 70% of data to the training set and the remaining 30% to the test set. Taking use of a training set three models were built using Microsoft Decision Trees, Microsoft Neural Network and Microsoft logistic regression. Later the test set was used to evaluate the accuracy of each model. Results returned by each comes next: 4.2.1 Microsoft Decision Trees which is a hybrid decision tree algorithm developed by Microsoft research and supports classification and regression tasks build the tree out of training data set. Results returned when the model was run on test set are shown by the confusion matrix in table 3.
Table 3: Decision tree performance on test data set Predicted Observed Accuracy Churner NonChurner Churner 1551 89.70% 151 15.79% 87.74% Non Churner 178 10.29% 805 84.20% Total 1729 956
The decision tree algorithm used for the prediction of churners returned 89.70 % true positives and 84.20 % true negatives which accounts for the overall accuracy of 87.74 %. 178 out of 1729 cases were predicted as non churners while they were churners. There were also 151 cases predicted as churners who were actually non churners. 4.2.2 Microsoft logistic regression algorithm recorded the results shown in table 4. This algorithm showed the overall accuracy of 89.01% with 91.03 % true positives and 85.35% true negatives International Journal of Computer Applications (0975 8887) Volume 9 No.7, November 2010 12 Table 4: Logistic regression performance on test data Predicted Observed Accuracy Churner NonChurner Churner 1574 91.03 % 140 14.6 % 89.01 % Non Churner 155 8.96 % 816 85.35 % Total 1729 956
4.2.3 Microsoft Neural Network algorithm is a feed forward network and uses back propagation to adjust the weights. Weighted sum approach is used for combination function. While tanh function is used as activation function for hidden nodes, sigmoid function is used for output nodes. Microsoft neural network does not allow more than one hidden layer and as stated below: Number of hidden neurons = c * sqrt (m*n) where n is the number of input neurons, m is the number of output neurons, and c is a constant. Number of nodes in a hidden layer can be changed by giving different values to c. This constant is supplied to the algorithm by giving value to hidden- node ratio. Different values were set to this parameter but the best performance was shown when this constant was set to 4. Table 5 shows the performance of the model. 90.74% cases were predicted as churners who were actually churners and 86.08% cases were predicted as non churners and were actually non churners, which accounts for the overall accuracy of 89.08% Table 5: Neural network performance on test data set Predicted Observed Accuracy Churner NonChurner Churner 1569 90.74 % 133 13.91 % 89.08 % Non Churner 160 9.25 % 823 86.08 % Total 1729 956
4.4 Accessing Models The results of three algorithms; Decision trees, Logistic regression and Neural Network is shown briefly in table 6. To meet the first and third objectives of this study we tried three techniques to get the best results but all the three models showed almost similar performance with logistic regression showing a little bit higher churner hit rate of 91.03%. Overall accuracy of neural network is slightly higher as compared to other two techniques. Table 6: Summery of results Decision tree Logistic regression Neural Network Churner hit rate 89.70% 91.03 % 90.74 % Non churner hit rate 84.20% 85.35 % 86.08 % Accuracy 87.74% 89.01 % 89.08 % In order to compare the over all performance of the three techniques used, it is customary to calculate other measures of accuracy as well. These measures are defined below: Sensitivity = True positive / positive Specificity = True negative/ negative Precision = true positive / (true positive + false positive) negtive positive negtive y specif icit negitive positive positive y Sensitivit Accuracy
Formula Figure :3.1 In the table 7 the three algorithms are compared as per the above defined measures. Table 7: Other measure of accuracy Decision tree Logistic regression Neural Network Sensitivity .897 .910 .907 Specificity .842 .853 .860 Precision .911 .918 .921 Accuracy .877 .890 .890
Next comes the lift chart which represents graphically the changes that a mining model causes. This chart is an ordinary tool to compare different models when the target is a discrete variable (such as a customer being churner or a non churner). The standard lift chart contains a single line for each model.
International Journal of Computer Applications (0975 8887) Volume 9 No.7, November 2010 13
Figure 4: Accessing the models 5. DISCUSSION AND CONCLUSION First and the third objective of this study were discussed in the above section. This section is mainly focused on the second objective of this study. As mentioned before, all the features used for the churn prediction were either demographic features or billing or usage features. Purpose of our second objective was to get an insight about the importance of these three types of features in the churn prediction. Having a look on the results returned by all the three techniques, we came to this conclusion that the demographic features have the lowest affect on the churn prediction. But it was not easy to get to the conclusion about the billing and usage features. It seems that these two types are equally important for predicting churn. Figure 5 is called the dependency network and depicts the importance of features. The number assigned to a feature shows the importance of that feature in churn prediction. Dependency network is the result of decision tree algorithm and preference given to the features is a bit different from other algorithms, but on the whole, Usage and billing features stand at the first place and demography has very little affect on the churn prediction. . Figure 5: Decision tree Dependency network Cluster membership incorporation also seems to be important as all the three algorithms have taken them in use which brought us to the conclusion that the clusters carry the significant information about the usage pattern. Ignoring the incorporated clusters during model building dropped the accuracy of all the three algorithms considerably. 6. REFERENCES [1] (2005). General Overview of the ICT in Iran. Tunis, World Summit on the information society (WSIS). [2] Au, T., Li, S., and Ma, G., (2003). "Applying and Evaluating to Predict Customer Attrition Using Data Mining Techniques." Journal of Comparative International management 6(1). [3] Au, W.-H., Chan, K. C. C., and Yao, X., (2003). "A Novel Evolutionary DataMining Algorithm With Application to Churn Prediction." IEEE Transactions on Evolutionary Computation 7(6): 532-545. [4] Baesens, B., Verstraeten, G., Van den Poel, D., Egmont- Peterson, M., Van Kenhove, P., and Vanthienen, J., (2004). Bayesian network classifiers for identifying the slope of the customer lifecycle of long life customers. European Journal of Operational Research 165: 508-523. [5] Berry, J. A., and Linoff, G. S., (2004). Data Mining Techniques For Marketing, Sales, and Customer Relationship Management. USA, Wiley. [6] Burez, J., and Van den Poel, D., (2006). "CRM at a pay-TV company: Using analytical models to reduce customer attrition by targeted marketing for subscription services." Expert Systems With Applications 101: 512-524. [7] Chae, Y. M., Hee Ho, S., Won Cho, K., Lee, D. H., and Ha Ji, S., (2001). Data mining approach to policy analysis in a health insurance domain. International journal for Medical Informatics 62: 103-111. [8] Chiang, D.-A., Wang Y.-F, and Lee, S.,-L., (2003). "Goal- oriented sequential pattern for network banking churn analysis." Expert System with Applications 25: 293-302. [9] Chiou, Jyh-Shen., (2003). The antecedents of customers loyalty toward Internet Service provider. Information & Management 41: 685-695 [10] Daskalaki, S., Kopanas, I., Goudara, M., and Avouris, N., (2002). "Data mining for decision support on customer insolvency in telecommunication business." European Journal of Operational research 145(10): 239-255. [11] Gappert, C., (2002). "Customer Churn in the Communications Industry." A KPMG LLP white paper, U.S member of KPMG International. [12] Hadden, J., Tiwari, A., Roy, R., and Ruta, D., (2005). "Computer assisted customer churn management: Stat-of- the-art and future trends." Computers & Operations Research 10(2): 139-158. [13] Han, J., Kamber, M., (2003). Data Mining: Concepts and Techniques, Morgan Kaufmann. [14] Hsieh, Nan-Chen., (2004). An integrated data mining and behavioral scoring model for analyzing bank customers. Expert Systems with Applications 27: 623-633. [15] Hung, S., -Y., and Wang, H.-Y., (2004). Applying Data Mining to Telecom Churn Management. Proceedings of International Journal of Computer Applications (0975 8887) Volume 9 No.7, November 2010 14 third annual conference, Pacific Asia Conference on Information Systems, Taiwan. [16] Hwang, H., (2004). "An LTV model and customer segmentation based on customer value: a case study on the wireless telecommunications industry." Expert Systems with Applications 26: 103-120. [17] Kim, H.-S., and Yoon, C.-H., (2004). "Determinants of subscriber churn and customer loyalty in the Korean mobile telephony market." Telecommunication Policy 28(6): 751- 765. [18] Lariviere, B., and Van den Poel, D., (2004). "Investigating the role of product features in preventing customer churn by using survival analysis and choice modeling: The case of financial services." Expert Systems with Applications 27(2): 277-285. [19] Li, S.-T., Shue, L.-Y., and Lee, S.-F., (2005). "Enabling Customer Relationship Management in ISP Services Through Mining Usage Patterns." Expert System with Applications 100(23): 1-12. [20] Madden, G., Scott J. S., and Cobel-Neal, G., (1999). "Subscriber Churn in Australian ISP Market." Information Economics and policy 11(12): 195-207. [21] Malhotra, R., Malhotra, D. K., (2003). Evaluating consumer loans using neural networks. International journal of management science 31: 83-96. [22] McKay, H., (2005). "Development of contractual relationship between an ISP and its customers - Is a fairer deal in sight?" Computer Law and Security Report 21: 216- 225. [23] Minguel, A. P. M. L., (2005). "Measuring the impact of data mining on churn management." Internet Research: Electronic Network Applications and Policy 11(5): 375-387. [24] Mitra, S., Acharya, T., (2003). Data Mining: Multimedia, Soft Computing, and Bioinformatics. USA, John Wiley & Sons. [25] Ng, K., Liu, H., (2001). "Customer Retention Via Data Mining." Issues on the Application of data Mining: 569-590. [26] Prinzie, A., Van den Poel, D., (2005). Incorporating sequential information into traditional classification models by using an element/position-sensitive SAM. Decision Support Systems. [27] Van den Poel, D., (2003). "Predicting Mail-Order Repeat Buying: Which Variables Matter?" Tijdschrift voor Economie and management 48(3): 371-403. [28] Van Den Poel, D., Lariviere, B., (2003). Customer attrition analysis for financial services using proportional hazard models. European Journal of Operational Research 157: 196-217. [29] Verhoef, P. C., Donkers, B., (2001). Predicting customer potential value an application in the insurance industry. Decision Support System 32: 189-199. [30] Wei, C.-P., and Chiu, I.-T., (2002). "Turning telecommunications call details to churn prediction: a data mining approach." Expert Systems with Applications 23(10): 103-112. [31] Yan, L., Wolniewicz, R. H., and Dodier, R., (2004). "Predicting Customer Behavior in Telecommunications." IEEE Intelligent Systems: 50-58. [32] Bart Lariviere and Dirk Van Den Poel (2004), Investigating the post-complaint period by means of survival analysis, Volume 29 , Issue 3,667-668 [33] Shen-Tun Li, shu ching kuo , Knowledge discovery in financial investment for forecasting and trading strategy through wavelet-based SOM networks, Expert Systems with applications, 34(2): 935-951 [34] Jyh-Shen Chiou (2004), The antecedents of consumers loytality towards Internet Service Provider , Information & Management, 41(6): 685 695 [35] Ding-An Chaing et. al. (2003), A recommender system to avoid customer churn , Expert systems with applications, 36(4): 8071-8075