Testing the Utility of the Neural Network Model to Predict History of Arrest among Intimate Partner Violent Men

Abstract

Risk assessments are typically based on retrospective reports of factors known to be correlated with violence recidivism in simple linear models. Generally, these linear models use only the perpetrators’ reports. Using a community sample of couples recruited for recent male-to-female intimate partner violence (IPV; N = 97 couples), the current study compared non-linear neural network models to traditional linear models in predicting a history of arrest in men who perpetrate IPV. The neural network models were found to be superior to the linear models in their predictive power. Models were slightly improved by adding victims’ report. These findings suggest that the prediction of violence arrest be enhanced through the use of neural network models and by including collateral reports.

1. Introduction

Psychologists and legal experts regularly make use of tools designed to predict criminal recidivism. Although these tools have made significant improvements in recent decades, there is general agreement that experts are poor at predicting which inmates will recidivate and which will not [1]. One reason why current methods for predicting recidivism perform poorly is that they treat all inmates as a homogenous group without considering type of crime or motivation for the criminal act [2]. Early research suggested that measures focusing on specific types of criminals using criminogenic theories may possess greater predictive power than models lacking in specificity [3]. Therefore, tools are needed that are designed to predict a specific type of violence recidivism in specific subpopulations of criminals, such as perpetrators of intimate partner violence (IPV).

Secondly, researchers generally use linear models such as logistic regression to predict recidivism. However, linear models are limited in their ability to predict complex phenomena such as recidivism [4]. For example, linear models assume that the independent variables are normally distributed and are not correlated although most psychosocial measures are correlated. Neural Networks (NN) are non-linear models that do not have some of the limiting properties of linear models. For example, they do not require that independent variables be uncorrelated, or normally distributed [5]. Thus, NN may have a greater ability than linear models to predict which men are likely to commit IPV in the future and which are not.

Third, models of recidivism have not typically taken both perpetrator and victim report into account. Although IPV has been shown to be better predicted by perpetrator characteristics than victim’s characteristics, the addition of the victim’s report of the perpetrators characteristics and history may enhance the model’s predictive power over models that rely on the report of only the victim or only the perpetrator.

1.1. Risk Assessment for Intimate Partner Violence

IPV is a complex and intractable public health problem that affects many people around the world [6]. It includes physical (e.g., grabbing, choking, punching), psychological (e.g., yelling, put-downs), and sexual (e.g., rape, coercion for sexual acts) abuse towards a former or current romantic partner [7]. Generally, IPV has been thought to be motivated by men’s need for power and control over their female partner [8]. In the United States, an average of 20 people per minute are victims of physical violence by a romantic partner, resulting in over ten million male and female victims within a given year [7]. According to the most recent national survey, nearly one in four women (22.3%) and one in seven men (14%) have been victims of severe physical violence by a romantic partner in their lifetime [9]. Homicide is one of the leading causes of death for women under the age of 44 with 3519 women murdered in the United States in 2015 [10]. In a review of femicides over the past decade, 55.3% were murdered by their intimate partners [10].

Among prison inmates, nearly one quarter (22%) have been convicted of family violence [11]. About 90% of offenders in state prisons for family violence had at a minimum caused physical injury to their victim and a staggering 28% of those offenders had murdered their partner [11]. Although recidivism rates are high for a wide variety of crimes, they are particularly high in cases of IPV [12]. Even after arrest and court-mandated interventions, between 15% and 60% of IPV perpetrators reassualt within three years [13,14,15,16,17]. Given the risk of serious injury and death that partners of violent men and women face, it is particularly important that we are able to distinguish those individuals who will be arrested and are repeat offenders of IPV. However, most researchers use linear methods to predict recidivism which have false positive and false negative rates that exceed 50%. New methods of predicting recidivism are not likely to improve the accuracy of the predictions without developing theories and tools upon which to build and assess predictive models [4].

1.2. IPV Risk Assessment Tools

With the recent advances in our understanding of IPV and its related factors, there has been a proliferation of measures designed to assess for the likelihood and severity of future incidents of IPV, including the Spouse Abuse Risk Assessment (SARA) [18]), the Domestic Violence Screening Instrument (DVSI) [19]), the Ontario Domestic Assault Risk Assessment (ODARA) [20]) and the Domestic Violence Risk Appraisal Guide (DVRAG) [21] The present study utilizes two commonly used measures of IPV risk factors; the Personal and Relationships Profile (PRP) [22], completed by the male perpetrator and the Danger Assessment Scale (DA) [23], completed by the female victim.

1.3. Neural Networks

Such risk assessment tools are generally entered into multiple regression models to predict arrest. Supervised NN models are similar to multiple regression models with the exception that they use a new class of nonlinear forms [24,25]. A NN is a mathematical simulation of a collection of idealized “neurons” and how they are connected. These models have a set of input neurons and a set of output neurons and which are not analyzed independently, but rather in the context of all variables entered with it to be processed as a whole [26]. One of the greatest strengths of NNs is their ability to adapt. Once the network designer sets up the basic parameters of the model, the model will examine the input and give a certain output. That output is then compared to the correct outcome and if the actual outcome does not match the desired output the model makes small adjustments to the connections between the neurons and tries again with the next piece of input. As this process is repeated, the model becomes increasingly adept at predicting the correct output based on the specific input it is given [5].

NNs are more flexible and may be better suited than multiple regression for the non-linear relationships often found in complex, real-world applications. For example, Pao [27] found that NNs were better able than regression models to predict debt ratio and identify important determinants of capital structure among various industries in Taiwan. Barcelos-Tronto, da Silva, and Sant’Anna [28] found that NNs outperformed linear models in predicting the amount of resources a manager would need to allocate to a particular project. In medical settings, NN models generally outperform their linear counterpart in predicting mortality and disease-free survival [29], although not always [30,31]. A meta-analysis of medical research suggests that NNs generally outperform regression models [32], especially in smaller samples (N < 5000).

NNs are particularly adept at identifying the kinds of adaptive and nonlinear systems found in biology and the social sciences [4], although they are more commonly used today in real estate appraisal, stock price prediction, and voice and image recognition software [33]. One study applying NN to emergency room data to identify victims of domestic violence [26] found that it could predict group membership with 78% sensitivity and 89% specificity. However, this study did not compare the accuracy of NN vs. linear models or examine the area under the curve (AUC), a standard measure of accuracy. Another predicting hospital violence found that the NN models were superior to linear regressions in terms of sensitivity, specificity and AUC [34]. Other researchers using both types of models to predict recidivism among offenders released from prisons [4] or psychiatric hospitals [35] found no improvement in NN models over and above linear models in predicting recidivism. However, in a larger study of offenders released from prison, Palocsay and colleagues [36] found that NNs were superior to multivariate logistic regressions in predicting overall recidivism. Thus, the research on the advantages of NN when applied to criminal justice data is mixed. No study to date has applied NN modeling to perpetrators of domestic violence.

1.4. The Current Study

The current study tested four models designed to identify perpetrators with a history of arrest. One linear model included only data collected from the perpetrator of IPV, while a second linear model added information collected from the partner of the perpetrator. Similarly, two supervised NNs were created, one lacking data from the partner and the other will include the partner’s data. To use the perpetrator-only model as an illustration, the NN will be given information on each of the relevant variables for a particular male. The model will rate the importance of each variable, guessing, for example, that a history of substance abuse is more important than a history of childhood abuse. The model will then guess as to whether that particular male has a history of arrest or not. If the model is correct then no adjustments will be made to the model. However, if the model is incorrect regarding his arrest history then minor changes will be made such as increasing the importance of violent history or reducing the importance of a history of childhood abuse. The model’s ability to accurately predict which men have a history of incarceration and which do not will then be measured via several comparison criteria discussed below. It is hypothesized that the NN models will be better able to predict history of arrest in perpetrators of IPV than linear models and that models that include victim report will outperform those that lack victim report.

2. Materials and Methods

2.1. Participants

Adult, heterosexual cohabitating couples with a history of IPV were recruited for this study through newspaper advertisements recruiting “couples experiencing conflict”. To meet inclusion criteria, women must have reported two or more male-to-female acts of physical violence in the past year on the Conflict Tactics Scale-2 (CTS-2) [37]. To meet criteria for the distressed/nonviolent group, women must have reported no violence in the past year, no serious violence ever, and a score of 5 or lower on a 7-point scale of relationship satisfaction [38].

Participants were recruited from a large city in the Southern United States. The advertisement specified that couples must be 18 years of age or older, married or living together as if married for at least 6 months, and be able to read and write in English (N = 92 dyads). Twenty of those couples failed to adequately complete the necessary measures leaving 72 couples to be included in the analyses. The mean age of the men was 32 with a range of 19 to 52 years of age. Nearly one-third of the participants reported they were unemployed at the time of the study. The mean income was approximately $30,000 per year with the highest earner reporting annual income of $130,000. The average length of time that couples had been in their current relationship was 6 years. The racial make-up of the sample consisted of African Americans (56%), Caucasians (26%), Latino (10%), Asian (6%), and those who selected “Other” (3%). Roughly half (56%) of the males reported having children. No participants reported having abused their children or of having knowledge of children being abused.

Of the IPV men, 60% reported a history of arrest and 10% had been arrested on a domestic violence charge. Seven percent reported having been arrested on both a domestic violence charge and a charge other than domestic violence. Nearly one-third of the males had a history of incarceration in either a jail or prison. Of those with a history of incarceration, the majority had only been incarcerated once. The participant with the greatest number of incarcerations had been in prison or jail 5 times. The shortest length of incarceration was 2 weeks and the longest was 192 months with a mean of 39 months.

2.2. Procedures

Data were collected as part of a larger study of IPV [39]. Female partners were screened over the phone using the CTS-2 [37] and the Short Marital Adjustment Test [38]. Couples where the women reported at least two male-to-female physically abusive acts in the past year were screened in. During the first 3 h session, only men completed questionnaires and participated in computer-based tasks. Men were paid $30 for participation in this session. During the second 3 h session, male and female participants completed questionnaires independently, including the PRP and DA. For safety, both members of the couple were separately interviewed and independently debriefed to answer any questions and to assess their present levels of anger, the partners were reunited and paid $35 each for their participation in the second session.

2.3. Safety Measures

This study was fully approved by the University of Houston Internal Review Board, Social Sciences Committee. In order to maintain the safety of the participants, safety procedures developed by Dr. Anne Ganley were used [40]. Following the assessment, participants were placed in separate rooms and debriefed to assess danger and safety. When necessary, safety plans were developed. All participants received referrals for community resources including, but not limited to, counseling services, hotline numbers, and shelters. Female participants were telephoned one week later to determine if their participation in the research project had caused any negative events. In no cases did women report violence due to participation in the assessments.

Measures

The Personal and Relationship Profile. Straus and colleagues [22]) created the comprehensive Personal and Relationships Profile (PRP) to be administered by researchers or clinicians as a stand-alone measure to assess risk factors associated with IPV. The PRP is a 187 item, 21 scale measure designed for research on intimate partner violence and as a screening tool in clinical settings. The PRP assess for 14 personal variables (antisocial personality, borderline personality, criminal history, depression, gender hostility, neglect history, post-traumatic stress, social desirability, social integration, substance abuse, stressful conditions, sexual abuse history, violence approval and violent socialization) and eight relationship variables (anger management, conflict management, communication problems, conflict, dominance, jealousy, negative attributions, relationship commitment and relationship distress). The items are scored 1 (strongly disagree), 2 (disagree), 3 (agree), 4 (strongly agree). The 21 subscales of the PRP have strong empirical support linking them to IPV. Straus [41,42,43]) examined the correlation of nation-to-nation differences in scores on eight PRP scales, providing strong evidence for the PRP’s concurrent validity with similar measures.

The Danger Assessment. The Danger Assessment (DA) [23] is a tool designed to assess the likelihood of serious injury or death occurring as a result of IPV. Unlike the PRP, which is completed by perpetrators, the DA is a violence risk assessment tool designed for female victims. Questions on the DA are designed to assess several risk factors associated with IPV such as the male’s history of violence and incarceration, his use of violence to obtain sex, access to weapons and his history of substance abuse. The measure also assesses relationship variables such his level of jealousy and the degree to which she believes he attempts to control her daily activities. It is also a good predictor of IPV recidivism and homicide. A retrospective validation study of the DA compared cases of femicide with cases of IPV in which the female victim was not killed. This study found that 90% of the cases included fell within the receiver operator curve (ROC) suggesting that the DA is adept at identifying cases of lethal IPV in relation to non-lethal IPV [43]. Roehl et al. [44] examined data provided by 1307 battered women and compared the test results of the DA with two other risk assessment questionnaires and the victims’ perception of risk. The DA had the highest correlation with subsequent abuse, although the correlation was small (r = 0.38).

The DA originally consisted of 15 items selected based on previous research on factors related to IPV as well as input from women in battered women’s shelters. Questions are designed to assess several risk factors associated with IPV such as the male’s history of violence and incarceration, his use of violence to obtain sex, access to weapons and his history of substance abuse. Other items more directly assess his level of violence by measuring whether or not his violence is increasing in severity, if he has choked his victim, his history of threats to commit suicide, and history of suicide attempts [44]. Item 14 of the DA is related to child abuse and was omitted from the DA as an endorsement of that item would mandate a report to Child Protective Services.

2.4. Comparison Criteria

When assessing the accuracy of recidivism prediction models, the most commonly used indicators are: (a) the false-positive rate (FPR) which is the proportion of non-recidivists incorrectly predicted by the model as recidivists, and (b) the false-negative rate (FNR) or the proportion of recidivists incorrectly predicted as being non-recidivists and (c) the percentage of total correct predictions (TCP) [4]. One limitation of FPR, FNR, and TCP is that these scores are influenced by the base rate of recidivists found in the sample. Thus, the FPR, FNR, and TCP do not generalize beyond the sample and are not particularly useful in cross-study comparisons of the effectiveness of prediction models [45].

In addition to the criteria employed by Caulkins and colleagues [4], Receiver Operating Characteristic (ROC) analyses were conducted. The ROC is a graphical plot of the true positive rate against the false positive rate for binary classifiers. The ROC analysis provides tools to compare various models and select those with optimal performance. A plot displays a diagonal line that marks chance classification as well as a curve marking the model’s classification. Larger areas under the curve (AUC) values represent higher levels of accuracy. This analysis calculates the sensitivity and specificity of each risk factor combination as well as the chances of correctly identifying those with a history of incarceration and those without. Sensitivity refers to the likelihood a test will produce a positive result when the condition is present (true positive) and specificity refers to the likelihood that a test will produce a negative result when the condition is not present (true negative). An AUC value of 0.80 or above suggests the model has good accuracy levels [46]. Good models also typically possess a sensitivity above 80% and specificity of 60% or better.

3. Results

3.1. Data Analysis

SPSS 20 was used to construct several NN models which varied in the number of hidden layers and the number of neurons in each hidden layer. The models were created using 70 percent of the data, and their accuracy in validation was tested on the remaining 30 percent of the data. Cross validation was used to prevent over-training of the model and minimize the generalization error. By partitioning the data into subsets, analyzing one subset and comparing the results to the second subset, the model is prevented from “over-learning” the training data.

Regression models were also created and tested with SPSS 20 using the Stepwise (Backward/Wald) method. The logistic regression model analyses included Hosmer–Lemeshow goodness of fit chi-square. The models were created using a stepwise method originally entering all 23 variables used in the NN model. Only four variables met criteria for inclusion in the model using men’s report only, conflict, sexual abuse history, relationship distress and substance abuse, chi-Square = 2.73 (df = 8, p = 0.950) (See

Table 1. Summary of Logistic Regression Analyses in model that did not include victim report.

Predictor	B	SE B	eB
Conflict	0.669	0.663	1.952
Sexual Abuse History	0.946	0.599	2.575
Relationship Distress	0.196	0.593	0.822
Substance Abuse	2.210	0.618	0.110
Constant	0.926	1.569	2.523

Note: e^B = exponentiated B. * p < 0.05. ** p < 0.01. *** p < 0.001. Hosmer–Lemeshow chi-Square = 2.73 (df = 8, p = 0.950)

). Five variables were included in the logistic regression adding women’s report on the DA scale (See

Table 2. Summary of Logistic Regression Analyses in model including victim report.

Predictor	B	SE B	eB
Conflict	0.669	0.664	1.935
Sexual Abuse History	1.032	0.614	2.807
Relationship Distress	0.173	0.599	0.841
Substance Abuse	2.205	0.632	0.110
Danger Assessment	0.204	0.233	0.816
Constant	0.947	1.574	2.578

Note: e^B = exponentiated B. * p < 0.05. ** p < 0.01. *** p < 0.001. Hosmer–Lemeshow Chi-Square = 4.18 (df = 8, p = 0.841).

), Chi-Square = 4.18 (df = 8, p = 0.841). Both linear models demonstrated an improvement over the null model.

3.2. Performance of Models

The performance of all four models across all four criteria can be found in

Table 3. Comparison of regression and neural network models.

Model	FPR	FRN	TPC	AUC
Neural Network Models
Men’s Data Only	19%	25%	85%	0.962
Men’s + Victim Report	17%	20%	85%	0.964
Logistic Regression Models
Men’s Data Only	39%	32%	65%	0.809
Men’s + Victim Report	34%	24%	69%	0.812

Note: FPR = false-positive rate; FNR = the false-negative rate; TCP = percent total correct predictions; AUC = area under the curve.

. Among FPR, FNR and TPC, the TPC value is most commonly reported in studies that compare classification models as it is the simplest to interpret. The TPC of the NNs is higher than the regression models, suggesting that NNs correctly classify a higher percentage of the participants than linear models. Although the FPR, FNR, and TPC are useful indicators of the model’s performance, they cannot be directly compared to determine which model was most effective. The Receiver Operating Characteristic is a graph used to illustrate and evaluate the performance of a model used in binary classification. The Area Under the Curve is a value that allows for direct comparison of the models by comparing the relative ratio of sensitivity to specificity for each model. It is essentially a measure of the model’s ability to classify a participant. As was hypothesized, NN models were more effective than the linear models in predicting history of arrest, and models that included victim report outperformed models that did not include victim report.

4. Discussion

When compared to linear models, the NNs were better able to predict which participants had a history of arrest, as evidenced by their superior AUC values. This suggests that non-linear models such as NNs may prove more useful in real-world settings when the goal is classification of complex phenomena. NNs may be particularly adept at predicting specific means of recidivism such as IPV as they have fewer of the limitations of linear models that were discussed earlier.

Models that included victim report outperformed models that did not include victim report, although by only a marginal amount. The current study found that the addition of a second source of information would enhance the model’s predictive power. For both the linear approach and the NN approach, the models that included victim report performed slightly better than models that did not include victim report. This suggests that adding victim report data may improve the ability of judges, parole boards etc. to predict who will commit acts of domestic violence in the future and who will not. It is likely that additional sources of information, such as parole officers, family members and others who know the perpetrator, will increase our predictive ability as well. However, involving victims of crime in the criminal justice system may not warrant the increased risk to their safety [47].

Tu [48] and others continue to argue that logistic regression is the clear choice when the goal of model development is to examine causal relationship among variables. This study contributes to the growing body of evidence suggesting that NNs are superior to standard linear models. There has recently been a call to develop models that incorporate both regression and NN models because a serious limitation of NNs is their tendency to over-fit the data during the training process, which limits the model’s performance during testing [28]. Regression models have less potential for over-fitting because the range of functions they can model is more limited. Therefore, hybrid models combining the linear and NN models may be preferable. For example, epidemiologists have developed models for newly diagnosed cases of liver disorders by combining the two approaches [49]. Microbiologists have combined linear and NNs to model Escherichia coli growth [50]. Attempts at combining the models focuses on using linear models to set parameter limits to constrain the NN and prevent over training. Future research in this area would do well to examine the strengths and weaknesses of the two approaches and design models that incorporate the best of both.

This study suggests that non-linear models that include corroborating reports, such as victims’ reports, may be more useful in clinical and forensic settings. Given the serious consequences of domestic violence, any improvement in our ability to predict recidivism may save not only money in legal and correctional costs but also lives. IPV causes tremendous physical and emotional pain, thus even modest improvement in prediction garnered by the use of these models may have dramatic impact. Better models may help the justice system to identify perpetrators likely to recidivate and give them longer sentences or more intensive interventions in order to protect families.

4.1. Limitations

A limitation of this study is that it is based on retrospective reports of arrests and incarceration. Ideally, one would conduct a prospective, longitudinal study of identified batterers and predict via NN models subsequent arrest and incarceration. Hagan and King [51] note that few studies have actually developed tools for predicting recidivism and tested them via longitudinal studies on former inmates. Like many previous studies, this study used an analogue design, using history of incarceration as a proxy for future recidivism. However, recidivism and history of arrest are not synonymous but are used interchangeably in the current study.

Another significant limitation is the reliance on self- and partner-reported history of arrest and incarceration as a proxy of recidivism in this cross-sectional study. Corroborating police reports would be useful as an adjunct outcome variable. Moreover, both perpetrators and victims may underreport the frequency and severity of violence in the home. However, this study was primarily a demonstration of the utility of non-linear modeling to predict group membership rather than an attempt to model recidivism.

A community sample of cohabitating couples was recruited for the current study and these findings may not generalize to shelter samples, court-mandated or incarcerated samples. Moreover, the small was relatively small. Although small sample sizes have been found to reduce the efficacy of both types of models, there has been research conducted that demonstrates both linear and NN models are equally impacted by small samples [52]. Thus, while the relatively small sample size may have affected model fit, it did not likely impact the relative performance of the models.

4.2. Future Directions

Additional studies should be longitudinal in nature, using clearly defined criteria for measuring recidivism. Longitudinal studies would allow for an analysis of how these predictor variables change over time and would add a layer of complexity to the prediction that would be well suited for non-linear models. In an applied setting, data could be collected from individuals recently released on parole and that data could be continuously added to the model, thus increasing the predictive power of the model as it has the most recent information regarding the parolee.

There are various types of NNs that have been developed, each differing in the method by which they arrive at a final means of classifying a participant. For example, Radial basis function networks and Kohonen self-organizing networks differ from the feed-forward network used in the current study. Additional research exploring these various types of networks and their ability to discriminate between recidivists and non-recidivists may be of further use to professionals who must make decisions regarding sentencing, probation, and parole.

This study suggests that additional sources of information may make significant improvements in model performance. Thus, future studies should examine which sources of information are most useful. While gaining collateral information aids the model, the cost of obtaining such information may outweigh the benefits. This study suggests that the advantages of nonlinear NN modeling outweighs any improvement in accuracy than simply adding collateral reporting to a linear model.