Analysis of two diverse nursing records applications: Mixed methods approach

The first healthcare computer-based information systems were used mainly for physician order entries, test results reporting, financial management and pharmaceutical and radiological activities support (1). Lately, healthcare organisations have been adopting information systems capable of storing and managing all clinical practice data. The integration of modern information and communication technologies (ICT) in patient care and the use of electronic health records (EHR) for health data management have shown some positive impacts on healthcare quality and patient outcomes (2, 3, 4). Nevertheless, ICT design should be adequately adapted for the specific clinical environment. In a study of 3900 hospitals in the United States, ICT implementation showed no positive effects on mortality, cost savings, adverse drug events or readmission rates. The reasons for this were attributed to a lack of organisational changes, poor adoption and poor utilisation among healthcare professionals (5). Efficient strategies for the implementation of user-friendly ICT solutions are crucial and should consider healthcare workers’ needs, to reduce their resistance to adoption and ensure optimal staff support for the best clinical results (6).

The nursing process is a fundamental method for planning and providing integrated nursing care, enabling nurses to identify patients’ needs and plan nursing care for those needs. The nursing process is cyclical and consists of four to six phases according to different sources. Our study used the five-phase nursing process in the theoretical background consisting of assessing, diagnosis, planning, implementing and evaluating (7). This definition of the nursing process establishes the nursing diagnosis as an individual phase.

The organisation NANDA International, Inc. (NANDA-I) defines the nursing diagnosis as a clinical judgment about individual, family, or community responses to actual or potential health problems or life processes, forming the basis of nursing care planning (8). The entirety of the nursing process is reflected in nursing documentation as documented nursing health history, nursing diagnosis, nursing care plan, nursing-related outcomes and planned interventions (6, 7).

Electronic nursing record (ENR) systems are designed for electronic documentation of nursing-specific data and should enable documentation of nursing care plans according to the nursing process. The nursing process contributes significantly to nurses’ participation in patients’ treatment, but in Slovenia, it is often limited to education and theory and is not adequately represented in contemporary clinical nursing documentation (9). It should be fully implemented in contemporary patient care and ENR systems (10), enabling an efficient implementation of the nursing process in clinical practice to improve the quality of healthcare, improve nursing documentation and support holistic interdisciplinary patient care (11).

New technology acceptance can be enhanced if a user-centred approach is used in product development and evaluation (12). A user-centred approach has been used before, to evaluate the perceived usability of diverse health-related electronic systems’ functionalities and user satisfaction (13, 14). It could also be employed to assess and improve the adequacy of nursing process methodology integration and usability of diverse ENR systems. An extensive review of software usability evaluation studies reported that the most commonly used methods are usability testing, heuristic evaluation and questionnaire methods, while also highlighting the importance and lack of studies using both qualitative and quantitative methods for evaluation of usability (15).

In this study, qualitative and quantitative methods were used to conduct a user-centred analysis of two Slovenian electronic documentation applications that differed in basic functionalities, graphical interface and nursing process incorporation. Our primary objectives were to analyse the opinions and attitudes of end-users regarding the usability of two systems and the ability of those systems to support the nursing process. Our secondary objectives were to generate a list of key system features to support usability and to demonstrate selected methods of evaluating systems.

Quantitative and qualitative methods were used to conduct a user-centred analysis of two diverse applications for documentation of nursing practice. To achieve an in-depth analysis of results an explanatory sequential mixed method was used.

Of 61 students, 53 were women and 8 were men, most of them using ICT every day (n=58; 95%). On computer-related skills, 56% of students rated their skills as “competent”, 26% as “advanced” and 18% as “beginner”. Most students had never used software designed for documentation of healthcare-related data (n=50; 82%). Among the students that completed the survey, 26 students used application B and 35 students used application A. No significant differences in demographic data were detected between the groups.

The user experience and perceived quality of the nursing process methodology integration were addressed in 47 statements rated by students. The aggregated results for the observed aspects of each application’s use are presented in Table 1.

Although application B was rated favourably in all aspects of application usability, students’ answers to open-ended questions showed different results. Participants were asked to identify the three best and three worst attributes of the application used. Students reported 258 attributes: 136 concerned application B and 122 application A. Figure 1 represents the number of reported positive and negative attributes for both applications, thematically categorised according to the questionnaire-based aspects of application usability.

Figure 1

The most frequently reported positive attributes of application B were the transparent overview of data and documentation speed, while the most frequent negative attributes were inappropriateness for older staff. Regarding the nursing care plan formulation, the students reported positive attributes that included promotion of holistic nursing care plan formulation, integration of preformed listed terms and simplicity of the formulation process. Reported negative attributes regarding the nursing care plan formulation included difficulty in navigating listed terms and scarce selection of listed terms. Additionally, participants reported some professional concerns regarding application B, which included inadequate individualization of a nursing care plan, inappropriate listings of preformed terms and inadequate patient data security.

The most frequently reported positive attributes of application A were nursing needs assessment transparency, while the most frequent negative attributes were documentation speed, outdated visual design and inability to correct documentation errors. Regarding the nursing care plan formulation, the students reported positive attributes that included integration of the nursing care process, promotion of holistic nursing care plan formulation and support of individualized formulation of nursing diagnoses. Only one negative attribute regarding the nursing care plan formulation process in application A was detected. A student reported difficulties formulating adequate nursing diagnosis using the problem, aetiology and signs/symptoms principle.

Unfavourably rated application A was included in additional in-depth analysis. In a qualitative analysis of students’ reflections, we used open coding thematic analysis of text to formulate codes, which were then grouped into eight code categories and three themes, as shown in Table 2.

The students were dissatisfied with the visual design and some functionalities of the application. Regarding visual design, the colour scheme seems to play an important role, as one of the students wrote:

“The programme would benefit from some aesthetic improvements since the outlook should feature livelier colours as opposed to the current grey design. I would recommend for example light blue, white or light green. Something unobtrusive but friendly to the user’s eyes.”

Besides poor visual design, the reported limitations of the application’s functionalities were focused on time consumption and the amount of manual typing required for documenting patient data. Both those aspects can be summed up by a student that wrote:

“In the majority of cases documentation of data requires manual input of nursing care plan elements, which takes the healthcare worker an enormous amount of time.”

Additionally, the qualitative analysis provided some information regarding the positive aspects of application A. In students’ opinions, the nursing process is suitably integrated into the inner workings of the applications, as two students wrote:

“The application does a good job of guiding the user through the stages of nursing documentation. All the parts of the documentation (anamnesis, needs evaluation, goals and so on) are clearly designed and the user is able to easily switch between different parts of the application.” “In my opinion, the application is easy to use since it is clearly structured, understandable and guides you through the nursing care process.”

Additionally, participants reported some professional concerns. Concerning reported inadequacies of the application, the students suggested a stricter use of standardized nursing languages. In their opinion, the use of preformed statements and standardized languages such as NANDA-I is beneficial for efficient documentation and promotion of quality care. As examples of this, students wrote that the use of preformed statements could decrease the time needed for documentation and improve the quality of documented data.

“What I believe is even more important is the implementation of NANDA diagnosis into the application, which would additionally ease users’ workload and save time.”

“We should consider abolishing the option of nursing diagnosis creation according to the problem, aetiology and symptoms/signs principle. Manual creation of nursing diagnosis is not suitable for in-depth statistical analysis and objective patient condition evaluation, since there is no universal way of interpreting data.”

As seen above the students recognized standardized nursing languages to be important in ensuring suitable formulation of patient data for research purposes.

The study participants comprised a relatively homogeneous student group that frequently uses ICT but rarely for healthcare-related data documentation. Although the students were in the final semester of a nursing science degree, a large percentage (82%) had never used any application for patient data documentation. This indicates the lack of implementation of nursing informatics in the Slovenian nursing curriculum, which should be addressed in response to the increasing importance of informatics competencies in a modern clinical environment (24). In their reflections students also reported the need for staff training as a recommendation for improved ENR implementation.

Application B provided a better user experience and was perceived more suitable for nursing care plan documentation, since according to our survey it was rated favourably in all seven analysed aspects of application usage. According to the questionnaire results, application B performed better in operability, learnability, attractiveness and understandability, factors that in literature characterise greater software usability (25). The application is visually well designed and enables data input with minimal manual typing, making the application fast and easy to use. Its visual design features a colourful user interface with large letters and tabs. Some patient data can be documented using interactive images, and vital signs can, without additional effort, be presented in charts or tables (22). On the other hand, the user interface of application A, last updated in 2005, is poor in visual design and requires an excessive amount of manual typing. Answers to the open-ended questions also pointed out the superior visual design of application B, which could be associated with an improved user experience and would explain the preference for it in the survey results.

Open-ended questions results show that students using application B were predominantly dissatisfied with the level of individualisation of the nursing care plan and reported inadequate collections of listed aetiological and diagnostic terms. This points to an incomplete implementation of nursing standardized languages, which are normally associated with an increase in nursing care plan quality (26, 27). Based on the participants’ answers to open-ended questions, we could assume a more satisfactory integration of the nursing process in application A. Students using this application reported predominantly positive attributes related to nursing care plan formulation. Previous research (19) on the use of electronic documentation application A showed that its design prompts the user towards consistent documentation of individualised aetiological and diagnostic terms related to nursing diagnosis statements. Although application B was more recently developed and represents a modern EHR system, we can speculate that its design focuses on simplicity of use, visual design and incorporation of user needs while not ensuring patient individualisation and adequate integration of the nursing process.

Qualitative analysis of written reflections further confirms that visual design and time needed for documentation significantly influenced the perceived professional suitability of the evaluated applications. Students recognized numerous positive aspects of application A, predominantly regarding the organization and presentation of nursing-specific data and nursing care plans. This leads us to believe that the overwhelmingly inferior survey score of application A could be a consequence of poor visual design and excessive need for manual typing. Specifically, students disliked the sharp edges, small font and monotonous colour scheme that made the overall appearance of application A look outdated. Time spent documenting was another concern regarding the application use, with students urging the use of preformed statements and integration of standardized nursing language for faster and easier formulation of nursing care plans.

A user-centred approach can be used to evaluate the user experience and quality of nursing process integration in diverse ENR systems. Furthermore, our study highlights the importance of using qualitative and quantitative approaches in software evaluation.

Firstly, diverse opinions regarding the aspects of evaluated software solutions are difficult to adequately include in a structured questionnaire. While the main aspects of the applications’ use were included in the structural parts of the questionnaire, some were not predicted. For example, in open-ended questions, the students using application B reported concerns about inadequate patient data safety. Students did not consistently give reasons for their concerns, but one of the students reported that enabled access to other hospitalised patients indicated a lack of patient data security. Such functionality is not enabled in application A, for which no such concerns were detected. This aspect of application use was not included in any item of the structured part of the questionnaire and could therefore not be subjected to quantitative analysis.

Secondly, the survey results could prove to be misleading. Answering open-ended questions, students using application B reported the inadequate scope for individualising nursing care plans, which was not evident in the rated statements of the questionnaire. Although the questionnaire included items regarding the applications’ usability for the creation of an accurate and individually appropriate nursing diagnosis, the students did not rate these aspects of the application significantly lower. The qualitative analysis of student reflections also confirmed that students were satisfied with multiple aspects of application A that include data organisation, holistic care promotion, nursing process integration and so on, however, this was not evident in the survey results. Again, the students did not rate the applications’ usability for nursing care plan formulation and quality of formulated nursing care plan significantly higher in comparison to items describing visual design in rated statements of the questionnaire.

This could indicate that students might have reported a relatively lower level of favourability for all aspects of application use, in response to the visually poorly designed and slowly working application, while in the case of application B assigning relatively higher levels of favourability in response to the novel aesthetically appealing design. This is supported by the undesirably high internal consistency coefficient of the entire questionnaire (Cronbach’s alpha=0.97). These concerns are a significant result of our study since they indicate that future user-centred evaluations of software solutions should be carried out and interpreted cautiously. This study also highlights the importance of empowering end-users for competent participation in the development and critical evaluations of future digitalization of the nursing process. Users may be unable to objectively evaluate ENR systems’ professional suitability if they are influenced by a poor or excellent software design and therefore provide misleading results. Open-ended questions and in-depth qualitative analysis should be included in future studies since they can provide relevant information regarding the professional suitability of evaluated software solutions that otherwise cannot be adequately addressed in a structured quantitative questionnaire.

The study limitations include the low number of participants and its educational setting. Participating students lacked clinical expertise and did not use the evaluated applications before the study took place. This made them unable to provide the best possible feedback while evaluating the software in a strictly controlled environment. Each student had access to a computer, was familiar with the patient beforehand and had no time restriction while documenting patient data. Our study could be improved if the students were placed in a simulated environment and interacted with the evaluated applications for a longer time. This could better mimic the intended use of the application in actual clinical practice by an experienced user. Future research should take place in a clinical environment among a larger number of experienced users. An important application usability aspect is usage in the actual clinical environment scenarios, where users may be faced with a lack of appropriate hardware, environmental disturbing factors and time constraints (28). Furthermore, participants in this study were limited to a single university. Nevertheless, this study offers important insights for future nursing documentation applications development.

We used a user-centred approach to compare two diverse applications for electronic documentation of the nursing process. The results of this study provide new information on the user-centred evaluation of nursing-specific applications for clinical practice documentation. Additional findings using qualitative analysis further show that future user-centred evaluations of software solutions must consider the possible inability of users to objectively rate professional suitability under the influence of a satisfactory or dissatisfactory experience using the evaluated software. Such studies could lead to the empowerment of healthcare professionals to participate in the development and critical evaluations of future digital solutions.