Solution to On-line vs On-site Work Efficiency Analysis on the Example of Engineering System Designer Work


 Day-to-day working activities have been heavily altered by COVID-19 pandemic, forcing a transition from traditional on-site work to on-line telework across the whole world. It has become much harder to efficiently organise, guide and evaluate employee’s work. There are different factors that can influence “work from home” quality, and many of these affect such work negatively. A set of relevant methods and tools should be developed which could improve this situation. The goal of the study is to summarise related background of this problem and to propose an approach to overcoming this problem. To achieve the goal, design engineer’s work is evaluated in an appropriate environment (e.g., AutoCAD, etc.) using automated analysis and visualization of IS auditing data.


I. INTRODUCTION
Considering the current global health situation and the relevant work organisation trends affected by COVID-19 (and in the future by other pandemics), it is clear that the emphasis on self-isolation and the transfer of work activities have a significant impact on the intensity of IT system use and habits for the performance of direct duties. It also has a significant impact on other aspects of users. The rapidly growing number of remote users, on the one hand, reduces the risks of spreading the epidemic, improves the overall environmental ecology (significantly reduces CO2 emissions and car congestion in cities) and maintenance costs (office space, etc.). On the other hand, it affects daily workflow efficiency, as well as complicates the management of companies and operational monitoring of the activities that users perform in the digital environment during the completion of their direct duties. Factors that may badly affect the efficiency of teleworkers include: • Decreased work motivation when working remotely and alone for a long time.
*Corresponding author's email: Vitalijs.Zabinako@abcsoftware.lv • Increased risk of the employee's "burnout" due to the prevailing feeling that one has to work alone, lack of direct support from colleagues and a common "spirit" of the team, "intrusion" into private life, which increases stress levels and enforces burnout in the long term. • Negative influence of various factors of private life, for example, performance of various household working activities, distraction by family members, etc., which in home working conditions begin to have a significant and negative impact on the performance of duties. • Limited opportunities to acquire new knowledge, especially for young employees with insufficient experience compared to working together and developing in the office, while working as a team. Therefore, there is a need to focus on automated tools and tools for employee efficiency assessment, the use and analysis of results of which do not depend on physical presence "on site in the office", as it takes place entirely in the digital environment. Such an approach is much more effective and even mandatory in a world affected by COVID-19 in particular, where the activities of IT system users are no longer directly observable. This inevitably calls for the inclusion of machine learning (ML) and data mining (DM) methods in the monitoring workflow of day-to-day users, in order to be able to assess staff efficiency, work capacity and motivation, as well as to perform assessment of new employee's knowledge because of decreased opportunity to receive face-to-face training of experienced colleagues and best practices in the use of IT systems.
The goal of the paper is to offer an automated solution that will make it possible to assess changes in the employee's performance if they work on-line or on-site. Thus, to achieve the goal, we need to determine the initial data that can be obtained automatically without additional work (and time) costs to understand how the data on the amount of work done should look like.
Thus, the research question of the paper is defined as shown in Fig. 1. The paper is structured as follows. The next section outlines background and related work in the respective knowledge domain. The third section is concentrated on obtaining and analysis of appropriate auditing data. The fourth section describes data visualization and application case. Conclusions are made on the result of current research and potential future research areas.

II. BACKGROUND AND RELATED WORK
The global shift to work from home has been going on for a long time, but it has become very relevant in early 2020 with the threat of COVID-19 infection. The pandemic has forced organisations to change their existing work structure and create new forms of teleworking. For example, in Europe, more than a third (37 %) of workers started working from home [1]. For some countries this percent is even close to 60 %, e.g., in Finland, which is the leader of teleworking (59 %), followed by Luxembourg, the Netherlands, Belgium, Denmark, etc. Global statistics shows that 57 % of company employees now work remotely full-time and another 16.5 % work remotely more than two-thirds of their time [2].
These new ways of working must provide a safe, productive and pleasant working environment. How to achieve, control and monitor this process? According to the McKinsey report, 80 % of those surveyed say that they enjoy working from home, while 41% say that they are more productive than before and 21 % are as productive as before. However, the question is -How can an employer actually assess whether an employee is really working more productively, or vice versa? Given the introduction of forced self-isolation and the significant tendency to relocate work to home and as remote as possible, employers now lack the ability to effectively monitor and control employee efficiency, including aspects such as the quality of task results and compliance with the time specified in the employment contract.
Given the prevalence of COVID-19 and the likelihood of its "recurrence waves", as well as the fact that several companies have assessed the economic benefits of converting work to remote work and reducing office maintenance costs, these problems and their aspects will become increasingly important over time.
Teleworking raises a number of issues that will need to be regularly monitored at the company's management level and addressed as necessary through employee burnout risk monitoring, additional motivation or training [2]. Among the biggest struggles with working remotely, the most influential on burnout are the following: • collaboration and communication (20 %); • loneliness (20 %); • not being able to unplug (18 %); • distractions at home (12 %).
If an employee works from home (isolated from the work environment and the team of colleagues), there is an increased risk that they may lose work motivation relatively quickly. Focusing on work in a home environment, where there are many distractions, requires much more effort and concentration. As an additional risk to reduce work efficiency when working from home, there may be an unsuitable workplace in a home or apartment environment, where there are many work-disturbing conditions.
Often, employees who work from home perform daily work and daily housework alternately, as a result of which the whole process of activity merges with side things from morning to the late evening. There is no logical division of labour and over time, for honest employees, such activities create additional burden and become unproductive. Problematic employees who lack self-discipline have problems with high quality and timely performance of duties.
Not every job is designed for working individually, on the contrary, most of the jobs require the involvement of the whole team, where mutual communication is important. Although there are a number of different Internet and mobile communication tools, they cannot completely replace the direct and indirect communication that is objectively formed when the whole work team is located in one office space. Insufficient communication, information "breaks", misunderstandings, etc. will certainly have a negative impact on productivity, although the employees feel that they spend as much time at work from home as when working in the office.
An additional problem also arises when hiring new and accordingly inexperienced employees. Their involvement in teleworking is difficult. It must be possible to monitor their activities and, if necessary, assist with training and the transfer of best practice.
Nowadays, a well-trained and motivated employee is one of the company's core values. The long-term decline in the ability of an experienced employee to work is a significant loss for the company. Although surveys and studies show that a large proportion of employees say that the opportunity to work from home is assessed very positively, in the long-run, there are negative effects that increase the employee's level of stress and may negatively affect their ability to work. Descriptions of adverse effects are widely available on several Internet resources.
For example, the employees state: • There is a feeling that you are working 24/7.
• It gets really lonely, work over time becomes boring.
• Work stress directly "breaks into a bedroom", as well as negatively affects health. • You stay "out of the discussion".
• You tend to be "slightly distracted".

F(x) Input data Output data
How it is possible to collect initial information and what kind of information?
Which data to expect for work efficiency analysis and how to visualize them?
What methods / algorithms is it necessary to use for processing the input data to obtain the output data Applied Computer Systems _________________________________________________________________________________________________2021/26 89 • There are many side factors at home that distract from work. • Independent activity requires more independence and self-motivation. • It is difficult to manage, and managers find it difficult to remotely monitor performance. • There are limited opportunities to form a team, there is no input from other people and no new ideas, there is no exchange of experience that classic employees receive when working in the office in teams. Since the situation with the pandemic has just begun, no real solutions have yet been proposed to apply, scientists and industrial companies are in search of appropriate solutions.
Here are some studies that are close in nature, which offer best practices [3] and recommendations [4]. However, they also have certain limitations, such as less formalized approach, and so far, cannot be fully used for telework evaluation purposes. Therefore, the question remains open and relevant for the future research [5].
"If a thing exists, it exists in some amount; and if it exists in some amount, it can be measured," Thorndike said in 1914. Work can also be defined in form of an object, which can exist in a certain quantity and can be quantified. It is possible to deduce that labour of any sort, and the extent can be measured by some methods. If we try to find such techniques in the domain of attempts to measure the productivity of a design engineer's work -most probably we will find a mixed bag of hypotheses, and nothing will be practical, or the results will be too theoretical, difficult and broad for engineer's work measurement and estimation.
As the application case of this paper, the estimation of work efficiency of engineering system designer is stated as a problem domain. The selection of this area is based on the fact that still we have metrics and standards of the evaluation of work amount of engineering system designer for their manual drawing development. As far as the CASE tools are often used for design of engineering system, the new metrics and methods for work efficiency evaluation should be established. The lack of methods is approved in a wide spectrum of literature resources. Some of them are [6]- [15].
When it comes to measuring designer's productivity (e.g., with AutoCAD usage), norms and standards were developed over the last century for manual drafting work evaluation, but there were no similar metrics for designer's productivity and efficiency measuring while using computer-aided design tools. A shift to a daily or weekly planning and control of regular activities can be used as an instrument to improve work efficiency and motivation. For this purpose, a database of frequently performed tasks will be created. As a result, the authors of the present research try to find out how to assess work of a designer, what tools can be used "as is", which of these can be adjusted for a new context, and what metrics / techniques can be made anew.
The problem of work efficiency estimation becomes more difficult because the engineer is dealing with multiple IS. A multi-system environment can be defined as a set of all IT systems / cloud services being used by the IT users in their work tasks, and this may result in the evaluation of the following internal organisational risks: 1. Not allowed usage of digital information that might lead to the leakage or loss of sensitive / confidential data via the usage of IT systems in the engineering sector. The intellectual property and expertise of a company are most susceptible. 2. User's inexperience in IS to for execution of their daily tasks. The authors of the present research propose a new technique for engineering system designers, which can be based on the construction of a typical behaviour model of users of IT systems for commercial objectives. The proposed approach is demonstrated with a designer's task estimation using AutoCAD [16] and task/project management software (in this case -LOTSIA PDM [17]). The research introduces a new technique for dynamic analysis of the actions of IT system users in a multi-system environment, while making typical user behaviour profiles for each system separately and monitoring each user from the point of view of a unified behaviour model for all IT systems.

III. APPROACH
Based on classical methods for work estimation [18], the methods that can be applied to assessing work are the following: • Time and motion research: o Direct and continuous observation of the task using a timing device to record the time required to complete the task. o Comparison with an etalon. Thus, one way or another, to measure the work in some business domain, where employees use IT systems for the performance of day-to-day duties, we should operate with activities, activity start / end time, and element content and concentration in the resulting artefact. A common concept for constructing an integrated behavioural model for the use of IT systems by engineering system designers is shown in Fig. 2.
So far, input data for work estimation of engineering system designers are the following: • Activities (with a link to a specific designer).
• Activity start / end time and therefore duration.
• Element content and concentration in the drawing.
Instead of manual observation of the designer's work, we propose automatically taking the log data from AutoCAD and estimating the time spent on carrying out certain activities mentioned in the previous section.
As part of the study, it has been found that there are no readymade auditing data solutions that take into account the time of the designer's activity. Therefore, it has been decided to develop a plugin that makes it possible to register all the designer's activities in working with AutoCAD and the time of their execution. Moreover, through the task management system used by the designers (in our case, LOTSIA), all activities can be associated with projects, specific tasks of certain performers and reporting periods for accounting hours. The plugin developed within the framework of the research for these purposes is in itself an additional valuable result that can be used for broader purposes by a wide range of specialists working with AutoCAD.
Audit data, which are obtained from LOTSIA and AutoCAD using implemented plug-in, contain all information required for the datasets mentioned above.
As output, the analyst of work efficiency can expect information about designer's productivity level, which is calculated based on hours registered as active working time by designer, commands performed during this active time multiplying them with their complexity coefficient received from domain experts. Moreover, a list of commands audited in the protocols of AutoCAD plug-in is analysed according to command popularity of usage by different designers. This will give us an ability to make a conclusion about designer's toolset in order to find designers missing some advanced commands in their practice and to organise the corresponding learning.
Application of sequential pattern mining algorithm [19] can provide an ability to define patterns of designer's work. These patterns can serve as a basis for definition of the set of specially selected patterns for the most effective designer's profile. The last one can be used for identification of non-typical behaviour of designers with the application of Markov's chain [20].
The output data give an ability to have a look on different metrics of engineering system designer's work, which are presented in form of tables and bar charts. The designer's workflows itself are visualized by a timing diagram (similar to Gantt chart [21]), which is implemented in a prototype of the solution that supports the proposed approach. Some data fragments obtained by the solution supporting prototype are demonstrated in the next section of the paper.

IV. APPLICATION CASE
The approach described in the previous section is applied to the analysis of engineering system designer's work. The sequence of activities for each designer is displayed as a workflow. Such workflows are presented for each user, for each created drawing file and for each tool or IT system used.
This form of displaying activity flows also makes it possible to identify breaks in the execution of activities. Depending on the specifics of a particular designer's work, it can be concluded, for example, whether these interruptions are justified or suspicious, given that the specifics of the designer's work provide for the so-called "doing" activities and "thinking" activities [22] are not recorded, but also to claim that nothing is done is unfounded.
In order to better evaluate designer's work in scope of timetable and activities of a particular project, appropriate data mapping can be performed for analysis purposes. This allows comparing designer's actual work with the panned workload and making justified assumptions about actual contributions to the project.
The list of IS used for day-to-day duties can be expanded with other tools, IT systems, browsers, chats, meeting platforms, document editors or e-mail tools and so on, which also take up working time and affect employee work amount.
Within the framework of this project, the solution for demonstrating the use of multi-systems has been tested on an experiment with the integration of two tools. Its successful implementation may justify the closure of other systems if it is not possible to automatically accumulate the necessary data on activities and their execution times in a certain format. However, format problems can also be solved by developing a special plugin; similar to the existing research, such a plugin is designed to collect data from the AutoCAD tool.
The timelines can be constructed for all the working days for all the employees and monitoring can be performed according to the information about working days spent on-line or on-site and making conclusions based on a comparison of the corresponding timelines. Fig. 3 shows the difference of such two days of engineering system designer.
The information about designer's workflows shows files (listed as drawings in Fig. 3) being worked on by different designers (listed as users in Fig. 3). Filled rectangles present workflows performed by a particular designer on a particular file at the particular time frames, which are listed on the top of   Fig. 4 shows two bar charts with the data about commands collected by a particular designer already multiplying them by their coefficient of complexity, hours worked with AutoCAD during the day and productivity. The list of commands performed by a particular user gives a basis to define the designer's tool set and the necessity to expand the set of commands used by one particular designer. The most popular commands used during the analysis period are shown in the chart.
The visual presentation of the output data gives a data set to which methods of artificial intelligence and machine learning can be applied, thus offering an ability to conclude about the following: To support the method, a prototype has been developed, which allows visualising the data according to the solution concept.

V. CONCLUSION AND FUTURE RESEARCH
The task of evaluating the work of an engineer, if we are talking about the work of a designer in AutoCAD, is not new, specialists of different levels have been examining this task with varying degrees of success for 50 years, starting from professionals working in this area and ending with academicians.
With a certain degree of confidence, we can assert that there are currently no real specific and software-supported methods for assessing the effectiveness of a designer's work in AutoCAD. The authors of the article have proposed a system prototype to support the method software. Fig. 4. Commands, working time and productivity metrics shown by bar chart. This paper describes a method for constructing of a unified common behaviour model for users of specialized IT systems in the engineering design industry, which is based on the analysis of multi-system user behaviour. The method uses automatically accumulated data from user activity log records registered in AutoCAD and LOTSIA tools as input data. As an expected result, visualization of the flows of activities of engineering system designers is supported by system prototype offered and user activities or series of activities worth highlighting and worth paying extra attention to. The method specification describes the problem environment, which is the activity of an engineering system designer working in a narrowly specific field, defines the conceptual scheme of the work efficiency evaluation solution and offers an ER diagram of potential method support IT solution design with its visualization. As a result, this document can serve as a basis for a specification for creating a system prototype.
The authors do not claim the result of applying this methodology as a conclusion that one designer is more effective, the other is less productive, our task is to collect data for analysis and propose visual presentation of output data metrics that a person who monitors designers could apply for their conclusions.
The following research results, in addition to practices for assessing effectiveness, can be considered: • Classification of engineering work.
• Systematization of scattered information (partially outdated) about the activities of the designer, which can be used not only to assess the effectiveness of their work, but also for other tasks. • A data model that can be used for engineering system designer's work evaluation. A plugin allows collecting data about activities, the time spent according to the developed data model.
At this stage of the research, the emphasis is placed on the development of a method that, using data from user activities and their execution time, is able to analyse the activities performed by users, measure the achieved results and evaluate their effectiveness. The method for determining user behaviour and operational efficiency solves the following main tasks: • Structures user activity log data automatically obtained from various systems used in the work of an engineering system designer. • Based on AI/ML algorithms, identifies user actions and their behaviour patterns and allows deducing their performance efficiency, such as execution time, atypically slow and frequent action sequences, atypical or unreasonable repetition of action sequences, interruptions in the work of users, number of users or statistical percentage, which have different characteristics, etc.
• Offers an approach to study the behaviour of users and technical processes of the system from the point of view of efficiency, using data from different sources, combining them in user profiles.
• Using the proposed approaches, identifies those user groups or technical processes that show relatively inefficient / unstable performance indicators of IT systems. The task of evaluating the work efficiency of an engineering system designer focuses on user behaviour. The expected results are indicated, which can provide the employee efficiency observation and monitoring of specialists at the engineering system design company with a basis for analysing user efficiency and dynamics, as well as comparing the performance of different activities to analyse and identify differences and typical shortcomings in their development. The main conclusions of this phase are as follows: 1. It is possible to use the advantages of AI/ML automation for integrated data of different IT systems in use for analysis and possible improvement of their work efficiency (which is especially important during COVID-19 pandemic). 2. Our examination of related works has allowed identifying potentially useful source/target data for this task and performing appropriate mapping, which can be used by UX experts in combination with AI/ML for guided well-grounded discovery of usability problems of each particular IS, based on the analysis of specific real use cases. 3. It is possible to continuously integrate an infinite number of systems into a common solution, provided that such use of the system can automatically generate data on users, their activities and execution time. The input data containing information on the activities performed by the user are sufficient to conclude on a certain set of performance criteria required for the management of the companies and the monitoring of effective / ineffective authorized / unauthorized activities.
Thereby, the research questions defined in the introduction (see Fig. 1) are answered with the results of the paper and can be expressed as shown in Fig. 5. Several areas for the future research are related to going in depth with output data, i.e., classification of timing activities according to their complexity, as well as going in breadth looking for compliance between designer activities and drawing

F(x) Input data Output data
How it is possible to collect initial information and what kind of information?
Which data to expect for work efficiency analysis and how to visualize them?
What methods / algorithms is it necessary to use for processing the input data to obtain the output data Data about user behavior and drawing content collected by AutoCAD plug-in Mathematical analysis, statistics and AI/ML algorithms for patterns mining and user profile definition.
Timelines of user activities performed in AutoCAD; Values of efficiency metrics presented by charts, tables, etc.
content changed under these activities. Continuing this study, it will be possible to perform practical definition of transformation of source/target data combinations and to make a suggestion of visual presentation of the analysis results (taking into consideration user's ability to tune-up obtained reports by data combination and comparison).