Evaluation of a virtual reality based interactive simulator with haptic feedback for learning NGT placement
Article information
Abstract
Background
The placement of a nasogastric tube is a blind process; the tube may be mistakenly inserted into other locations, leading to possible complications or fatal incidents. Effective education and training of nursing students to perform this procedure is essential.
Purpose
To investigate the application of the virtual reality based simulator with haptic feedback to nursing students for their learning around nasogastric tube placement.
Methods
A quasi-experimental non-equivalent group pre- and post-test study, in which the outcome measures of two classes of pre-registration nursing students were compared for their evidence of learning about the advanced NGT simulator (in addition to usual training) against the control group who only used mannequins for their usual education and training.
Results
There was a decrease, though remaining at a good level, in the technology acceptance rating within (p = .000) and between (p < .05) the simulator group than the control over time at post-test. Taking into consideration of some demographic differences at baseline between the two groups, analysis of results demonstrated that there was no predictor effect of those factors in relation to technology acceptance (F = .02, p = .922), but in scores for the test using multiple-choice questions (MCQ) about knowledge in nasogastric tube insertion (F = 23.4, p = .000). Both groups demonstrated significant increases in MCQ scores at post-test, with higher scores in the simulator group at both pre-test (p < .05) and post-test (p = .000). There was no significant difference in learning outcomes around competence in skills within the evaluation results between groups at post-test. Usability of the simulator system as rated by the simulator group was good.
Conclusions
Students did not reject the use of the new simulator for their learning about nasogastric tube placement. As an adjunct to conventional teaching and learning, the use of the simulator appears to be promising in enhancing the education and training of nursing students for development of the important clinical skill of safe nasogastric tube placement. Future studies are warranted, with the design inclusive of equivalent groups and a larger sample size to further the evidence in substantiating the use of this simulator for better learning outcomes.
INTRODUCTION
Nasogastric tube (NGT) placement is a common clinical procedure through which a flexible tube is inserted from a nostril into the stomach of a patient for feeding or drainage. The placement is a blind process given that the tube may be mistakenly inserted into other locations, leading to possible complications or fatal incidents (Curtis, 2013). Effective education and training of nursing students to manage risk and perform this procedure is essential. Computer-based simulation can provide a realistic and immersive virtual environment for teaching and learning (Lai, Luo, Zhang, Huang & Rozelle, 2015; Lateef, 2010). Research has indicated that this kind of simulation is in general an effective tool for teaching and learning skills related to health care (Harder, 2010; Roy, Lee, Chung & Park, 2013).
Conventionally, NGT insertion techniques are initially practised on low-fidelity mannequins, and subsequently human subjects. The prototype of a virtual reality - (VR-) based and haptic simulator for NGT insertion (Choi, He, Chiang & Deng, 2015; and Choi, He, Chiang, Deng & Qin, 2016) was created to facilitate the education and training of NGT placement for nursing students. The system is equipped with physical makeup of a NGT and human head, and an interactive graphic user interface (GUI) that provides high-fidelity and realistic simulation of tube insertion with unique feedback forces through the built-in haptic device (Picture 1). The feedback forces are developed based on the force models of finite element method (FEM) and fuzzy logic. The simulator is also capable of providing feedback on safety issues through the gag and coughing reflexes, and capacity to respond to verbal prompts to “swallow”, when the NGT is being advanced to particular anatomical positions during practice sessions. This advanced stimulator has the potential for improvement of NGT insertion education and training for nursing and other health professional students.
The VR-based simulator with haptic feedback for learning nasogastric tube placement
BACKGROUND
As with all educational approaches there is often a gap between the desirable implementation of the principles and the actual practice. However, when engaging curriculum renewal involving PBL the authors as facilitators of the change processes aimed to develop rigorous and sustainable models of that philosophy and methodology which maintained the integrity of the PBL principles in its implementation.
The purpose of the study was to investigate the application of the VR-based simulator with haptic feedback to nursing students for their learning about NGT placement. The objectives were to
1. evaluate the acceptance and usability of students to the system; and
2. examine the learning outcomes, the effectiveness of teaching with the system.
METHOD
Design
The study used a quasi-experimental non-equivalent group pre- and post-test design, in which the outcome measures of two classes of pre-registration nursing students were compared. It was hypothesized that there would be a significant difference in skills competence in the class of students who learnt processes and procedures with the advanced NGT simulator, in addition to usual education and training, when comparing with the control group who only used mannequins for their usual education and training.
Setting and participants
Convenience sampling within two separate classes of nursing students in a local university was performed to form the intervention and control groups respectively. The students were enrolled in fundamentals of nursing subject which included the learning about NGT insertion. Students in the control group were organized to use the advanced NGT insertion simulator freely after the formal study period. They had equal access to use of the new simulator. No one was deprived of learning about NGT insertion due to the study processes.
The intervention group involved an entire class of Bachelor of Nursing (BSN) students (2-year articulation programme) enrolled in the relevant subject for Semester 1 2016/17 (n = 40). The control group was an entire class of pre-registration Master of Nursing (MN) students who enrolled to the same subject in the same semester (n = 39). A NGT insertion practice laboratory session was scheduled for each class. The control group was wait-listed and free to use the simulator for practice after the post-test and formal study period.
Instrument
A set of evaluation questionnaires which had been used for a similar study (Pun, Chiang & Choi, 2016) were adopted for this study. Demographic information was collected for gender, age, academic qualifications, work experience in nursing, prior experience of practising NGT insertion, experience in using computer simulation, and frequency of participants’ self-directed practice in laboratory for NGT insertion after respective intervention and control group training.
A knowledge test on NGT insertion (20 multiple choice questions - MCQ) was constructed to determine knowledge about NGT insertion before and after the education and training. In addition to the NGT Insertion Skills Competence Checklist, this was another tool used to evaluate acquisition of knowledge. To ensure content validity, the knowledge test was reviewed by an expert panel of four experienced nurse educators in the school. They were asked to rate the relevance of each question in the MCQ test using a 4-point scale from ‘1’ (very irrelevant) to ‘4’ (very relevant). A rating of ‘3’ or ‘4’ shows content validity. The scale content validity index (universal agreement) / S-CVI/UA was .91, and mean item CVI / I-CVI was .97 (Polit & Beck, 2006).
The skills competence evaluation for NGT insertion was performed by the subject teachers with the NGT Insertion Skills Competence Checklist, which was modified from the one developed and used by the School for handson performance on a static mannequin. The checklist evaluated the performance of NGT insertion skills in terms of the correctness of technique, sequences, and manipulations made by a student during the processes of insertion. A 5-point Likert scale was adopted for the questions, with ‘1’ indicating ‘unable to perform the step’ and ‘5’ indicating ‘performed the step accurately’.
The Technology Acceptance Questionnaire (TAQ) was developed based on the Technology-Acceptance Model (TAM) for this study to evaluate students’ acceptance of the NGT insertion education and training with the advanced simulator as well as the usual mannequins. The tool aims to explain what causes users to accept a technology and their behaviors when using the system (Holden & Karsh, 2010; Pai & Huang, 2011). The TAQ in this study consisted of 13 items to collect the opinions of the participating students from four aspects, 1) perceived usefulness (four items); 2) perceived ease of use (four items); 3) intention to use (two items); and 4) attitude toward using the related training technology (three items). A 7-point Likert scale is used in the TAQ, with ‘1’ referring to the strongest agreement with an item and ‘7’ to the least. Lower scores indicate more positive student comments. Written comments from the students were also collected at the end of the questionnaire.
The IBM designed Computer System Usability Questionnaire (CSUQ) was used to evaluate intervention group students’ perceptions of the advanced NGT insertion simulator as a computer-aided training system. Like the survey of demographics, the TAQ and CSUQ are self-report questionnaires. The CSUQ has high reliability and validity and contains seven items with a 7-point Likert scale, with ‘1’ indicating ‘strongly agree’ and ‘7’ ‘strongly disagree’. The intervention group completion of the CSUQ enabled the researchers to investigate general usability system effectiveness and efficiency, ease and comfort with use and in learning. Overall, each self-report questionnaire (demographics, TAQ, and CSUQ) required approximately 10 minutes for participants to complete.
Procedures
Ethical approval was sought via The Hong Kong Polytechnic University’s Human Subjects Ethics Application Review System (HSEARS). During the first subject lectures, the two classes of BSN and MN students were fully informed about the study and the use of the advanced NGT simulator for laboratory learning and practice. It was anticipated that he entire class membership would participate as either the intervention or usual training group.
Prior to the NGT insertion practice laboratory sessions for both groups, a pre-test that contains the 20 MCQ, demographics sheet, and TAQ was completed by all the students. During the two weeks after respective practice laboratories of the two groups, all students were encouraged to perform self-practice with the advanced stimulator or usual mannequins according to their respective group allocations. Subsequently, students in each group were assessed by subject teachers with the skills competence evaluation checklist through a laboratory skill test arranged separately for each group. The control group repeated the MCQ and TAQ after the skill test as the post-test, while the intervention group repeated the MCQ and TAQ plus the CSUQ. Wait-listed laboratory sessions were provided to students in the control group in other weeks after the post-test so that they did not miss a chance to have their practice with the advanced NGT simulator. The flow of study is illustrated in Diagram 1.
Flowchart of the study
Data analysis
Descriptive statistics were applied to results of the MCQ, demographics, skills competence, TAQ, and CUSQ for the pre- and post-tests. Any differences in demographics between the two groups of students were identified. The MCQ and TAQ results obtained from the pre- and post-tests of respective groups were analyzed with split plot ANOVA (and post hoc as necessary) for any interactions, and within and between group differences. Alpha of the inferential statistics was set as < .05. The primary outcome measure of skills competence of NGT insertion was evaluated by comparing the results of skill test provided by subject teachers at post-test for any significant difference between the two groups, with significant demographic differences adjusted as necessary.
Ethical considerations
As mentioned, ethics clearance was obtained prior to the study. Students in the targeted subject within the BSN and MN programmes had been fully informed about the purpose and procedures of this evaluation study and the introduction of an advanced NGT stimulator for teaching and learning in the subject. Names of students were not exposed for dissemination of the study results and the anonymity and confidentiality were assured by using aggregated data without names for analysis. The collected data and questionnaires were stored in a locked cabinet and only the study team members were able to assess the data. Students who chose not to take part in completing the evaluation measures did not have any penalties imposed, nor changes induced to their learning activities in the subject. Written consent was obtained subsequently if students agreed to participate in the study. Risk of harm for taking part in the NGT learning activities of the intervention and control groups was minimal, and justice to the control group for their chance of practising with the advanced stimulator after post-test was assured by the wait-list arrangement.
RESULTS
All but one student in the two classes completed the pre- and post-tests; a student in the control submitted blank questionnaires, and was excluded from the analysis. Missing data were substituted by mean values of the group for the related items. Two demographic characteristics ‘age’ and ‘learnt / practised NGT insertion’ were found to be significantly different between the two groups (p =.000) at baseline (Table 1).
Characteristics of the participants
Taking these two factors into consideration, results of analysis by general linear model (repeated measures) demonstrated that there was no predictor effect of these two factors to TAQ (F = .02, p = .922), but MCQ (F = 23.4, p = .000) to the two groups. Both groups demonstrated significant increase of MCQ scores at post-test (Figure 1), with a higher score in the simulator group at both pretest (p < .05) and post-test (p = .000) compared to the control group (Table 2). The results may be associated with the differences in age and previous learning / practice experience of NGT insertion at baseline.
MCQ by group over time
Effects of the VR-based simulator with haptic feedback for learning of NGT placement
For the NGT Insertion Skill Competence at post-test, there was no statistical significance observed between the test scores of simulator (40.1, SD = 5.9) and control groups (41.3, SD = 5.2). The TAQ between groups were different at baseline (p < .05), with better technology acceptance for the simulator group (2.5, SD = .716) than the control (3.1, SD = .944). There was a decrease in TAQ rating respectively within (p = .000) and between (p < .05) the simulator group than the control group over time at post-test.
A significant time by group interaction effect of TAQ (F = 16.6, p = .000) was detected (Table 2). Although this suggested a statistically significant decrease in technology acceptance over time when compared to the the control group, the TAQ level was still good by rating at post-test (3.7, SD = 1.2) (Figure 2).
TAQ by group over time
Other results of different outcome variables are provided in Table 2. Usability of the simulator system was good as indicted by an overall rating as ‘agreed’ from the CSUQ scale of 1 -7 (with 4 as ‘neutral’; mean of rating = 3.5, SD = 1.4).
DISCUSSION
Summer, Gonzalez, Jimeno, and Christensen (2009) found that a computer-based software simulation system developed and used to complement the standard training of NGT insertion significantly increased the confidence of students in their ability to perform the procedure. However, students preferred hands-on learning first, prior to use of the software simulation system. In this regard, the haptic feedback of the new VR-based simulator applied in this study may compensate the lack of realism in merely using the computer-based software simulation system for education and training. In another study, the use of a high-fidelity simulator with gagging and coughing responses (Grady et al., 2008) for the training of NGT insertion demonstrated that students welcomed the use of simulators capable of providing realistic feedback and responses for their skill learning and practice. A strength of the VR-based simulator used in study is that it incorporates both the haptic feedback and VR responses to the learners; it fulfils the need for realism for their best learning experience and outcomes. Standardized simulation with high-fidelity simulation and computer-enhanced virtual reality offers protected opportunities for students to achieve mastery in a safe environment (Carson, et al., 2015).
Good technology acceptance (3.7, SD = 1.2), though decreased from the baseline (2.5, SD = .716), and good usability (3.5, SD = 1.4) of the VR-based simulator with haptic feedback were observed from the simulator group. The decrease in technology acceptance may reflect a realistic adjustment of perceptions in the simulator group after actual learning through the use of the simulator. In contrast to the similar technology acceptance in the control group at baseline (3.1, SD = .944) and post-test (3.1, SD = .923), this appeared to be purely a result of impressions of the system as the control had not actually used the system at the time of post-test. From the results as a whole, one can see that high-fidelity simulation continues to be well accepted by students as a mode of skill learning and development. The development of a well accepted and user-friendly high-fidelity simulator is essentially founded on successful interdisciplinary teamwork between the nursing and engineering disciplines (Summer et al., 2009). In particular, when the haptic feedback is incorporated in the system as a requirement to strengthen realism of actual practice, this fulfils the learning needs of students in a safe environment.
Age and the experience of learnt / practised NGT insertion were not found to have a predictor effect to technology acceptance of the new simulator in this study, but impacted on a significant increase in MCQ scores at post-test (F=23.4, p =.000) in the two groups. The control had more learnt / practised NGT insertion experience whereas the simulator group was more mature by age at baseline. Nevertheless, MCQ scores and skill competence evaluation results of the control (with more learnt/practiced NGT insertion experience at baseline) were not better than the simulator group. The reason is unclear and this might be related to the greater maturity and age of the simulator group who are more committed to becoming knowledgeable about NGT placement, and they are relatively less anxious during the skill evaluation test after a more comprehensive learning experience with the new simulator. A larger scale study with randomization for better sampling of equivalent groups at baseline may be able to demonstrate more reliable results. However, a randomization control trial to be conducted during the actual teaching and learning of students in a particular subject at this institution is considered as unethical (unjust) to the students. If time allows during the academic semester, a wait-listed cross-over design with randomization and bigger sample size could contribute to substantiation of results from this study.
Limitations
The results of analysis demonstrated that there was no predictor effect of age (given those more chronologically mature in the control) and learnt / practice of NGT insertion (given more experience in learning / practice of NGT insertion in the simulator group) to technology acceptance, but an impact of a significant increase in MCQ scores in the two groups. There is a threat to generalization of this study given the risk of sampling bias without randomization of the subjects. Although students in the simulator group were strongly encouraged to use the new simulator for further (two weeks) practice after a laboratory session made available to learn and practice NGT placement, additional practice with the simulator was not mandatorily organized. Levels of practice may be insufficient in inducing better learning outcomes (Chiang & Chan, 2014).
CONCLUSIONS
The limited feasibility of randomizing students for this study, may have led to a sampling bias associated with the differences in age, past experience of ‘learning / practicing NGT insertion’, and TAQ rating at baseline between the two groups. While there was no significant difference detected for MCQ and NGT insertion skill competence between groups at post-test, the simulator group indicated a good CSUQ rating (3.5, SD = 1.4), and maintained a good rating in TAQ (3.7, p = .000) after use of the simulator. Students did not reject the use of this VR-based simulator with haptic feedback for learning about the NGT placement. The use of this simulator appears to be promising in enhancing the training and learning experience of nursing students for the important clinical skill of nasogastric tube placement. Given the findings that the simulator was not rejected and not considered un-usable by students, and with no negative impact to the NGT knowledge and skill development as reflected by the MCQ and skill competence results, the simulator can be considered for regular use as an adjunct to conventional training. Future studies are warranted but with the design of equivalent groups and larger sample sizes to provide more evidence to substantiate the use of this simulator in pursuit of better learning outcomes.
Acknowledgements
This project was funded by the Halla-Newcastle PBL Education and Research Center, Research Funding 2016