|Year : 2019 | Volume
| Issue : 2 | Page : 105-111
Predicting physicians' satisfaction with electronic medical records using artificial neural network modeling
Hana M Alharthi
Department of Health Information Management and Technology, College of Public Health, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia
|Date of Web Publication||13-Sep-2019|
Dr. Hana M Alharthi
P.O. Box 2435, Dammam 31441
Source of Support: None, Conflict of Interest: None
Background: Studies involving the measurement of physicians' satisfaction with electronic medical records (EMRs) using a branch of artificial intelligence known as machine learning is rare in Saudi Arabia. Most of the studies have relied on traditional statistical methods. This study focuses on comparing an artificial neural network (ANN) model with linear regression model to predict physicians' satisfaction with their EMR. Aims: This study aims to compare the performance of ANN versus logistic regression (LR) modeling in predicting physician satisfaction with their institution's EMRs and compare sensitivity analysis results for both models. Methodology: Data were collected through a self-administered survey that was distributed to physicians working in inpatient departments at a major Saudi Arabian hospital (360-bed capacity). Using machine learning software, ANN and LR models were developed to compare performance power and identifying factors affecting physician's satisfaction through running sensitivity analyses. Results: The analysis included 115 physicians who answered the survey. ANN model produced a more accurate prediction when compared to the prediction produced by LR. ANN correctly classified the instances with 86.09% accuracy, compared to LR, which achieved 82.61% accuracy. In addition, sensitivity was higher in ANN model (0.86) compared to LR model (0.83). Specificity was lower in ANN (0.39) compared to the LR model (0.44), and the receiver operating characteristic curve was higher (0.79) for ANN, (0.76) for LR. ANN model identified three factors affecting physician's satisfaction: System integration with workflow; system features to enable physicians to perform their work well; and training. Conclusion: The results show the ANN model performed better than LR due to its nonlinear characteristics and discovered three new factors affecting physician's satisfaction. Therefore, ANN model should be used in physician's satisfaction prediction studies.
Keywords: Electronic medical record, electronic record systems, linear, logistic regression, neural network, nonlinear, physician satisfaction, Saudi Arabia
|How to cite this article:|
Alharthi HM. Predicting physicians' satisfaction with electronic medical records using artificial neural network modeling. Saudi J Health Sci 2019;8:105-11
| Introduction|| |
Electronic medical records (EMRs) are becoming increasingly prevalent worldwide. They are used to enhance standardization and accuracy in documenting patient medical history, visit summaries, prognoses, diagnoses, clinician notes, laboratory orders, test results, imaging results, past and current medications, allergies, and e-prescription information, among other things. EMRs also enhance the efficiency of interacting with other agencies for research purposes or in support of regulatory tasks. However, physicians often express frustration and dissatisfaction with EMRs, inhibiting more widespread adoption.,,,, Past studies have reported on specific aspects of EMR technology and services that physicians are having issues with including maneuvering within the system's many screens, the increased burden for documentation that takes away from face-to-face time with patients, and a general lack of usability.,,,,, Other studies note that a lack of training can have a significant impact on both user satisfaction and overall efficiency, further adding to the burden., In some cases, physicians have deemed EMR systems to be completely ineffectual, failing to align with clinical workflow, and promote coordinated care.,,
Using modeling technique is helpful to predict physicians' satisfaction. Many studies in the literature designed to predict physicians' satisfaction have used traditional statistics, such as regression, to generate their prediction models.,,,,, However, the major disadvantage of such methods is that they are based on linear structures within which the correlation between independent variables and the dependent variable is examined one by one in a linear fashion as the rest of the variables are held constant. However, satisfaction in general has highly nonlinear characteristics. Not all variables remain constant when analyzing the relationship between one independent variable and the dependent variable. Thus, accurately modeling satisfaction requires a more sophisticated approach that can handle more complex mathematical calculations. Machine learning techniques, for example, handles nonlinear modeling that more traditional statistics are not equipped to process. Indeed, other studies have used machine learning techniques, such as an artificial neural network (ANN), to model nonlinearity and capture the true rules within the underlying dataset. Interestingly, ANN has been used in studies based on datasets from many diverse fields, including business, marketing, and financial transportation, medicine, and communication.,,, However, to the best of the author's knowledge, no one has yet applied ANN to model physician satisfaction with their EMRs systems in Saudi Arabia. Thus, our objectives were threefold. First, we set out to develop models based on an ANN and logistic regression (LR) classifiers that predict physicians' satisfaction with their EMR system. Second, we compare the performance of each model to determine the best option for this particular application. Finally, we demonstrate proof of concept using sensitivity analysis to identify the specific factors that most influenced physician satisfaction.
| Methodology|| |
Data were collected through a self-administered survey that was distributed to physicians working in inpatient departments at a major Saudi Arabian hospital (360-bed capacity) between March 30 and May 25, 2010. Survey distribution was supervised by the medical director and information technology (IT) department. In this hospital, physicians are required to use the EMR system to document diagnosis, medical history, and laboratory and radiology orders. They also use this system exclusively to review test results and view alerts for medication interactions.
Survey design was based on the DeLone and McLean model. This type of survey is commonly used in the business sector to measure the success of information systems and has also been used previously to measure physicians' satisfaction with EMRs. Survey questions were based on a previously validated survey. The dataset under consideration here was used in a previous study, in which the author participated. The reliability of each item was evaluated using Cronbach's alpha. All values were all above >0.82, showing satisfactory reliability. Face validity of each item was determined by the research team, including a practicing physician and informatics experts. A pilot test was conducted of the survey that was distributed to 10 physicians. After the pilot, some of the questions were reworded and rearranged to improve clarity.
The final survey included 15 questions, divided into three categories: (1) quality of system performance; (2) quality of system information; and (3) service quality [Table 1]. In addition, physicians were asked about their demographics and their overall satisfaction with the EMR system. Each question asked for a response using a five-point Likert scale, ranging from “strongly disagree (1) to “strongly agree” (5). The overall satisfaction scale was collapsed, such that responses of “1,” “2,” or “3” were grouped as “not satisfied” and responses of “4” or “5” were grouped as “satisfied.” The survey was distributed to 220 physicians, of which 115 responded. For this project, only questions related to physicians' answers about the quality of the system, the quality of its information, and quality of services by IT department were used, and demographics were disregarded.
|Table 1: Satisfaction attributes represented by the survey with top attributes affecting physicians' satisfaction with their electronic medical records selected by Waikato Environment for Knowledge Analysis function|
Click here to view
TheWaikato Environment for Knowledge Analysis (WEKA) software, which was developed at the university of Waikato in New Zealand, is used as the data mining and machine learning tool. WEKA is used widely as a data mining tool for research in both the business and academic sectors. It supports many machine learning techniques, such as classification, regression, clustering, and summarization. For this project, an ANN and LR have been used to classify the dataset into respondents that were “satisfied” or “not satisfied.” In cases of missing data, i.e., some survey questions left blank, the WEKA “ReplaceMissing” function is applied to replace all missing variables/attributes data with the mean values from the training set.
Training and testing
The ten-fold cross-validation strategy was used to avoid overfitting. In this strategy, the dataset serves to both train and test the model. WEKA randomly divides the data into 10 subsets and randomly assigns one subset as the “test” set, using the remaining nine to train the model. This process is repeated nine times, thus producing 10 models. The mean and standard deviation of the performance indices of these 10 models is calculated to generate the final model.
Two types of prediction models have been used: an LR model and an ANN model. The performances of the models were compared using the following metrics: accuracy of prediction, sensitivity, specificity, and the receiver operating characteristic (ROC) curve. In addition, sensitivity analysis has been applied to each model, to determine the variables that are most significantly affecting satisfaction. The models are briefly described below:
Logistic regression modeling
LR is typically used when the target variable is categorical. For example, LR can be applied to predict outcomes, for example, “has cancer” versus “does not have cancer,” or “patient is living” versus “patient is deceased.” The model works by mapping the relationship between two variables and comparing the distance of the data points from the regression line. For the purposes of this study, LR analysis was applied to predict overall satisfaction based on 15 independent variables/questions.
Artificial neural network modeling
A neural network classifier known as the multilayer perceptron was applied using the default setting in WEKA architecture. This default architecture uses back-propagation with one input layer, a single hidden layer, and one output layer. The input layer consisted of 14 input nodes (survey questions). WEKA then generated a hidden layer that included 8 neurons as shown in [Figure 1]. The output layer predicts whether a respondent is satisfied or not satisfied, based on the inputs. The number of hidden layer neurons is generally determined based on trial and error. Similarly, there is no standard number of hidden layers. Whereas several studies have demonstrated that one hidden layer is sufficient to model complex nonlinear patterns, others have argued that more hidden layers are preferable. In this project, we found model performance was optimized with one hidden layer (models using 2, 3, and 4 layers were also tried).
Models performance evaluation measures
To evaluate model performance, we used the following metrics: prediction accuracy, sensitivity, specificity, and ROC plot. Prediction accuracy is calculated as:
TP + TN/TP + TN + FP + FN
Where TP, TN, FP, and FN refer to true positives, true negatives, false positives, and false negatives, respectively. Sensitivity is the probability of correctly identifying physicians who are satisfied. A higher sensitivity means a higher true positive rate and indicates that the model is better suited to detect satisfaction among physicians. Specificity is the probability of mistakenly identifying physicians who are not satisfied. A higher specificity means a lower false-positive rate and indicates the model is better suited to correctly detecting satisfied or dissatisfied physicians. The following are the equations used in this study, Sensitivity = TP/(TP + FN) Specificity = TN/(FP + TN). Another useful method for evaluating model performance is the ROC curve, which is a plot of the true-positive rate (0–1.0; y-axis) versus false-positive rate (0–1.0; x-axis). A curve close to 1 indicates the model is a good fit for prediction.
Sensitivity analysis has been applied using the wrapperSubsetEval function in WEKA. In this way, we optimized the method for finding the most important factors influencing physicians' satisfaction with EMRs. Here, ANN shows the relationship or correlation between the dependent variable and every single independent variable where all other variables are not held constant, on the other hand, regression shows the relationship or correlation between the dependent variable and an independent variable when all other variables are held constant. WEKA returns a list of the most influential factors for both classifiers in this study.
| Results|| |
The purpose of this study was to find out which modeling method is a better predictor for physicians' satisfaction with their EMRs. We compared the results of an LR approach to those generated by applying ANN classifier. Input variables included the answers to 15 survey questions, derived from a DeLeon framework, that were designed to discover physicians' opinions about the quality of EMR system performance, the quality of the information provided by the system, and the quality of services provided during and after system implementation. In this work, WEKA was used with no modifications to its default properties for both LR and neural network.
We found that the neural network-based model better predicted overall physician satisfaction compared to LR [Table 2]. Specifically, the neural network model showed higher accuracy (86.09% vs. 82.61%) than the LR model, and higher sensitivity (0.86 vs. 0.83). In addition, the false-positive rate was lower for the neural network model (0.39) versus LR (0.44). Finally, we found that the operating characteristic curve (ROC; a general metric of model performance) was more favorable (closer to 1) for neural network model (0.79) versus LR (0.76) when examining satisfaction outcome.
Using WEKA (wrapperSubsetEval function), we ran sensitivity analysis to show which variables according to how prominent each was as a predictor of overall physician satisfaction. The neural network selected seven attributes, and LR selected five attributes. There is no ranking involved. Both classifiers have presented the same four attributes that affect physicians' satisfactions, (1) the system is easy to use; (2) system performance is reliable; (3) the information is up to date; and (4) implementation process was acceptable. However, the neural network has identified three other attributes that LR ignored. The system is integrated with my workflow; systems features allow me to perform my work well, and the level of raining is acceptable. One factor identified by LR and ignored by neural network is information is available when needed. [Table 1] summarizes the selection.
| Discussion|| |
Although many studies have reported on physicians' satisfaction with their EMRs, nearly all have relied on traditional statistics based on linear modeling. In this research work, we have built to models using ANN and linear regression and compared their results to see which one a better fit for our problem under study is. We found out that the neural network did moderately better than LR.
In comparing the performance of both classifiers, we found that the neural network model was more accurate, sensitive, and has better specificity. For accuracy, 86% versus 82%, respectively, for ANN and LR. This is consistent with other studies comparing the two models. In one study, authors compared the performance of a neural network model versus LR in predicting satisfaction among patients hospitalized for lumbar disc herniation and found that the neural network model was (96%) accurate versus (94%) for LR. In another study, a neural network model was more accurate in predicting the occurrence of acute pancreatitis (induced portal vein thrombosis): (87.7% vs. 78.5%) using LR. And in a final, dramatic example, one study reported that a neural network model outperformed LR in predicting the accuracy of successful surgery for patients (95% vs. 41%). However, one study found there is no difference in performance power between neural network and LR.
In our study, we found that sensitivity and specificity were better in the ANN model (86%; 83%) and (39%; 44%) than in the LR model, respectively. This aligns with another study in which the authors aimed to predict the development of acute graft-versus-host disease after stem cell transplantation. They found that sensitivity and specificity were better in the neural network model (83%; 86%) versus LR model (22%; 90%).
We also found that the neural network model performed better than the LR model in terms of ROC (0.79; 0.76), respectively. ROC is considered a major performance evaluation tool as it reflects model performance without considering class distribution or error cost. Again, other studies have noted the superior performance of neural network models based on this metric. For example, one group reported that a neural network-based model for predicting kidney transplantation failure returned a ROC of 0.88, compared to an LR model who's ROC was 0.75. In another study, a neural network-based model predicted the treatment outcome of chronic hepatitis C patients with a ROC of 0.85, higher than the LR model (ROC = 0.58).
The higher performance of the ANN model lends greater confidence to its outputs. In this case, the outputs are the list of factors that most influence physician satisfaction with their institution's EMR system. To understand the factors that are important in determining physician satisfaction; very few applied more sophisticated, nonlinear methods, such as machine learning models, to identify important factors. The latter approach is important in this case to see if data mining using machine learning can identify factors that traditional statistics such LR cannot discover. In our research, ANN has identified three factors affecting physicians' satisfactions LR ignored. The system is integrated with my workflow; systems features allow me to perform my work well, and level of raining is acceptable.
Nonlinear classifiers can capture the underlying complexity between patterns and relationships. ANNs can easily find the underlying rules by “learning” what drives physician satisfaction or dissatisfaction. The model achieves this by mapping the relationships between physicians' answers to the fifteen questions (independent variables) and their response to the overall satisfaction question (dependent variable). Answers are mathematically analyzed in a nonlinear fashion, i.e., no variable is constant, but rather, all variables are involved in the calculations at the same time. This is more aligned with the complex ways humans come to their conclusions; driving factors can often be surprising and/or nonintuitive as the relationship between factors and overall satisfaction is nonlinear. Thus, a model like this can be critically important when trying to understand what drives opinions and relationships among people. For example, in one study, researchers were interested in how the extent to which hospitals invest in IT impacts hospital productivity. In this case, the results showed that productivity is also affected by other investments (non-IT labor and non-IT capital). The nonlinear relationships among these type of investments were exposed by this type of predictive analytics, and would arguably not have been identified through more traditional linear models.
ANNs use a series of internal weights and functions to define relationships between input and output variables. Some in the field raise concerns over the lack of transparency this approach lends to data analysis, especially compared to LR modeling in which it is relatively easy to trace the underlying equations.,, The accuracy and value of neural network models have been demonstrated and validated extensively in the literature and across many different fields. Moreover, it is often the case that practitioners in the health-care sector are primarily concerned with actionable results. Software packages, like WEKA, make it possible for decision-makers to more easily identify where the greatest potential for change lies. Thus, there is growing potential for approaches like these in the future, as even nontechnical personnel can gain tremendous benefit.
| Conclusion|| |
Our neural network classifier highlighted three areas affecting physician satisfaction that LR ignored: (1) system integration with workflow; (2) system features supporting physicians' work; and (3) training level. Thus, in this hospital, and for this specific EMR system, improvements in these three areas should be considered when trying to improve physician satisfaction. One recent report from the RAND Corporation suggests physicians at other institutions may feel similarly. However, more work is needed to be able to understand whether these factors are common across hospitals and/or EMR systems. The particular impact of the results of the present study is that the neural network modeling approach can be easily adapted at almost any institution. Thus, we hope that this work will encourage more studies of physicians' satisfactions with their EMR systems, particularly those applying more complex analysis through machine learning algorithms to more clearly reveal nonlinear relationships. This information can be critical for hospital personnel who are trying to develop more efficient and focused efforts with a high likelihood of success. Moving forward, such efforts will be key to enhancing the implementation and adoption of EMR systems worldwide.
The author would like to thank the original team for their efforts to collect the data, Salma Radwan, Sukainah Al-Muallim, and Zainab Tuwaileb.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Nguyen L, Bellucci E, Nguyen LT. Electronic health records implementation: An evaluation of information system impact and contingency factors. Int J Med Inform 2014;83:779-96.
Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: A review of the recent literature shows predominantly positive results. Health Aff (Millwood) 2011;30:464-71.
Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res 2010;10:231.
Vishwanath A, Scamurra SD. Barriers to the adoption of electronic health records: Using concept mapping to develop a comprehensive empirical model. Health Informatics J 2007;13:119-34.
Heyworth L, Zhang F, Jenter CA, Kell R, Volk LA, Tripathi M, et al.
Physician satisfaction following electronic health record adoption in three Massachusetts communities. Interact J Med Res 2012;1:e12.
Wing DA. The future of obstetrics and gynecology: MACRA, electronic health records, and more. Clin Obstet Gynecol 2017;60:840-52.
Jaspers MW, Peute LW, Lauteslager A, Bakker PJ. Pre-post evaluation of physicians' satisfaction with a redesigned electronic medical record system. Stud Health Technol Inform 2008;136:303-8.
Hudson D, Kushniruk A, Borycki E, Zuege DJ. Physician satisfaction with a critical care clinical information system using a multimethod evaluation of usability. Int J Med Inform 2018;112:131-6.
Koppel R. Is healthcare information technology based on evidence? Yearb Med Inform 2013;8:7-12.
Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, et al.
Enhancing patient safety and quality of care by improving the usability of electronic health record systems: Recommendations from AMIA. J Am Med Inform Assoc 2013;20:e2-8.
El-Kareh R, Gandhi TK, Poon EG, Newmark LP, Ungar J, Lipsitz S, et al.
Trends in primary care clinician perceptions of a new electronic health record. J Gen Intern Med 2009;24:464-8.
Fernandopulle R, Patel N. How the electronic health record did not measure up to the demands of our medical home practice. Health Aff (Millwood) 2010;29:622-8.
Dastagir MT, Chin HL, McNamara M, Poteraj K, Battaglini S, Alstot L, et al.
Advanced proficiency EHR training: Effect on physicians' EHR efficiency, EHR satisfaction and job satisfaction. AMIA Annu Symp Proc 2012;2012:136-43.
Ho CH, Wene HC, Chu CM, Wu YS, Wang JL. Importance-satisfaction analysis for primary care physicians' perspective on EHRs in taiwan. Int J Environ Res Public Health 2014;11:6037-51.
Unni P, Staes C, Weeks H, Kramer H, Borbolla D, Slager S, et al
. Why aren't they happy? An analysis of end-user satisfaction with electronic health records. AMIA Annu Symp Proc 2016;8:215-20.
Emani S, Ting DY, Healey M, Lipsitz SR, Karson AS, Bates DW, et al.
Physician beliefs about the meaningful use of the electronic health record: A follow-up study. Appl Clin Inform 2017;8:1044-53.
Niazkhani Z, Pirnejad H, Berg M, Aarts J. The impact of computerized provider order entry systems on inpatient clinical workflow: A literature review. J Am Med Inform Assoc 2009;16:539-49.
Shanafelt TD, Dyrbye LN, Sinsky C, Hasan O, Satele D, Sloan J, et al.
Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016;91:836-48.
Babbott S, Manwell LB, Brown R, Montague E, Williams E, Schwartz M, et al.
Electronic medical records and physician stress in primary care: Results from the MEMO study. J Am Med Inform Assoc 2014;21:e100-6.
Ryan MS, Shih SC, Winther CH, Wang JJ. Does it get easier to use an EHR? Report from an urban regional extension center. J Gen Intern Med 2014;29:1341-8.
Pollard SE, Neri PM, Wilcox AR, Volk LA, Williams DH, Schiff GD, et al.
How physicians document outpatient visit notes in an electronic health record. Int J Med Inform 2013;82:39-46.
Wright E, Marvel J. Electronic health records: Postadoption physician satisfaction and continued use. Health Care Manag (Frederick) 2012;31:259-67.
Love JS, Wright A, Simon SR, Jenter CA, Soran CS, Volk LA, et al.
Are physicians' perceptions of healthcare quality and practice satisfaction affected by errors associated with electronic health record use? J Am Med Inform Assoc 2012;19:610-4.
Somers MJ, Casel JC. Using artificial neural networks to model nonlinearity: The case of the job satisfaction-job performance relationship. Organ Res Methods 2009;3:403-17.
Carlucci D, Renna P, Schiuma G. Evaluating service quality dimensions as antecedents to outpatient satisfaction using back propagation neural network. Health Care Manag Sci 2013;16:37-44.
Zeinalizadeh SA, Shariatmadari M. Modeling and analysis of bank customer satisfaction using neural networks approach. Mark Intell Plann 2015;6:717-32.
Chong YL. A two-staged SEM-neural network approach for understanding and predicting the determinants of m-commerce adoption. Expert Syst Appl 2013;4:1240-7.
Nguyen DH, Patrick JD. Supervised machine learning and active learning in classification of radiology reports. J Am Med Inform Assoc 2014;21:893-901.
DeLone WH, Ephraim RM. The DeLone and McLean model of information systems success: A ten-year update. J Manage Inf Syst 2003;4:9-30.
Alharthi H, Yousef A, Radwan S, Al-Muallim S, Zainab A. A physician satisfaction with electronic medical records in a major Saudi government hospital. J Taibah Univ Med Sci 2014;3:213-8.
Hall EF, Holmes G, Reutemann P, Witten I. The Weka data mining software: An update. Assoc Comput Machinery 2009;1:10-8.
Sharma AB. A research review on comparative analysis of data mining tools, techniques and parameters. Int J Adv Res Comput Sci 2017;7:523-9.
Stefan RM. A comparison of data classification methods. Proc Econ Finance 2012;3:420-5.
Wang G, Lam KM, Deng Z, Choi KS. Prediction of mortality after radical cystectomy for bladder cancer by machine learning techniques. Comput Biol Med 2015;63:124-32.
Rathi M. Regression modeling technique on data mining for prediction of CRM. Inf Commun Technol 2010;10:195-200.
Tu Lack V. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 1996;11:1225-31.
Somers MJ. Using artificial neural networks to model nonlinearity: The case of the job satisfaction-job performance relationship. Organ Res Methods 2009;3:403-17.
Chang CD, Jiang BC. Using data mining techniques for multi-diseases prediction modeling of hypertension and hyperlipidemia by common risk factors. Expert Syst Appl 2011;5:5507-13.
Fawcett T. An introduction to ROC analysis. Pattern Recognition Lett 2006;8:861-74.
Matis GK, Chrysou OI, Silva D, Karanikas MA, Baltsavias G, Lyratzopoulos N, et al.
Prediction of lumbar disc herniation patients' satisfaction with the aid of an artificial neural network. Turk Neurosurg 2016;26:253-9.
Fei Y, Hu J, Gao K, Tu J, Li WQ, Wang W, et al.
Predicting risk for portal vein thrombosis in acute pancreatitis patients: A comparison of radical basis function artificial neural network and logistic regression models. J Crit Care 2017;39:115-23.
Azimi P, Benzel EC, Shahzadi S, Azhari S, Mohammadi HR. The prediction of successful surgery outcome in lumbar disc herniation based on artificial neural networks. J Neurosurg Sci 2016;60:173-7.
Chang YJ, Yeh ML, Li YC, Hsu CY, Lin CC, Hsu MS, et al.
Predicting hospital-acquired infections by scoring system with simple parameters. PLoS One 2011;6:e23137.
Caocci G, Baccoli R, Vacca A, Mastronuzzi A, Bertaina A, Piras E, et al.
Comparison between an artificial neural network and logistic regression in predicting acute graft-vs-host disease after unrelated donor hematopoietic stem cell transplantation in thalassemia patients. Exp Hematol 2010;38:426-33.
Witten EF, Hall A. Data Mining Practical Machine Learning Tools and Techniques. Burlington (MA): Morgan Kaufmann Publishing; 2011.
Tapak L, Hamidi O, Amini P, Poorolajal J. Prediction of kidney graft rejection using artificial neural network. Healthc Inform Res 2017;23:277-84.
Wang CH, Mo LR, Lin RC, Kuo JJ, Chang KK, Wu JJ, et al.
Artificial neural network model is superior to logistic regression model in predicting treatment outcomes of interferon-based combination therapy in patients with chronic hepatitis C. Intervirology 2008;51:14-20.
Larasati A, Camille DY, Slevitch L. The application of neural network and logistics regression models on predicting customer satisfaction in a student-operated restaurant. Proc Soc Behav Sci 2012;65:94-9.
Shmueli G, Koppius O. Predictive analytics in information systems research Management. MIS Q 2011;3:553-72.
Cui QA, Wang X, Li HJ, Kang XU. Using PCA and ANN to Identify Significant Factors and Modeling Customer Satisfaction for the Complex Service Processes. IEEE 18th
International Conference on Industrial Engineering and Engineering Management; 2011. p. 1800-4.
Shin DH, Biocca F. Explicating user behavior toward multi-screen adoption and diffusion: User experience in the multi-screen media ecology. Internet Res 2017;2:338-61.
Grnholdt L, Martensen A. Analysing customer satisfaction data: A comparison of regression and artificial neural networks. Int J Mark Res 2005;2:121-30.
Friedberg M, Peggy G, Kristin R, Busum V, Frances M, Aunon F, et al
. Factors Affecting Physician Professional Satisfaction and their Implications for Patient Care, Health Systems, and Health Policy. USA: Rand Health; 2013.
[Table 1], [Table 2]