Digital transformation of training programs for public administration

. The pandemic changed the paradigm of education at all levels. From primary schools to universities, teachers and students must conform to new patterns for teaching and learning. The goal of this paper is to share the experience resulted from the pilot project developed at the National Institute of Administration in order to deliver online training for public administration in the pandemic context. The project entitled PPP online (Romanian abbreviation for “online training programs”) was a challenge which supposed the redesign of the curricula and finding the best alternatives for delivering knowledge and interaction between trainers and participants. We must emphasize that there was only very limited experience in online training for Romanian public administration until 2020. In this paper we expose the choices considered for the transition to online training and the results of a study about participants’ perception regarding difficulties they encountered and the quality of programs. In this regard, we collected and analysed the feedback from more than 1000 participants (respondents to an online questionnaire). Our study has an optimistic conclusion about changing teaching and learning through technology in public administration. It can be appreciated that both trainers and participants can rapidly adapt to the conditions of the online learning environment and the proposed structure of training programs was validated by the results and positive feedback. We also present some best practices achieved during the pilot project.


Introduction
The purpose of the research is to analyse a model for organizing training programs implemented at the National Institute of Administration (INA) and how the participants and trainers have adapted to the online training environment, in order to validate a training program structure and identify best practices for future activity.
The paper contains a short literature review on learning management systems, the research methodology, a proposed structure for online training programs, an analysis of feedback questionnaires and the authors' opinions regarding the validation of the training model in the results and discussion section. In the conclusions section, the authors justify the validation of the model based on the results obtained by participants and the answers to the feedback questionnaires.
National Institute of Administration (INA) is a public institution which delivers specialized and short-term training programs for the personnel from central and local public administration.
Five Territorial Training Centres for Continuous Training in Public Administration function within INA, located in: Constanța, Craiova, Iași, Sibiu, Timișoara.
INA's role is to implement the strategic directions of the Government regarding the efficiency of public administration by professionalizing the personnel through training services provided by over 300 trainers registered in Institute's data base.
In 2018, INA organized 85 training programs with a total of 1215 participants, and in 2019, 200 training programs with 3020 participants. Expectations for 2020 were very high especially in view of the upward trend in previous years and positive feedback from the beneficiaries. However, the beginning of 2020 brought a totally unforeseen situation: the outbreak of the pandemic and the imposition of quarantine throughout the country. INA organized face-to-face training programs (participants and trainers were present in the same time in the training room), which in the light of the new restrictions could not be possible. The Institute was forced to adapt its activity to the new conditions, taking into account that the trainers had little experience in online training and the potential participants came from heterogeneous environments regarding the level of digitization. The only advantage of the Institute was an IT infrastructure under development and the fact that the first steps had already been taken in implementing an online registration system for training programs and in setting up a blended learning platform for sharing training materials and assessment.

Literature review
Scientific research on online learning has grown in the last decade with the development of technologies. Topics such as e-learning and blended-learning are covered in numerous scientific articles. We focused our research especially on defining online learning concepts and choosing the best solution for implementing online training programs.
Previous attempts of implementing blended learning at INA were defined as a mixture of online training materials and opportunities for interaction online with traditional face to face classroom methods. In the context of the pandemic, we were forced to move all training activities into a digital environment. The crisis has generated the need to implement virtual learning environments for all organizations involved in educational and training activities. A virtual learning environment (VLE) is a virtual space that brings technology and content together for e-learning training. It is a website that involves web platforms and content. It encompasses both course management and virtual classrooms so activities can be developed and students can access them to learn (Rossel, 2018). Al-Ajlan (2008) defined VLE as a form of e-learning that enables on-line interactions of various kinds to take place between teachers and students. VLE offers for institutes a number of benefits, such as access anytime and anywhere, better integration of application technology tools, opportunities for independent learning, improved motivation, and access to novel learning styles (Al-Ajlan, 2008).
As we mentioned, there was a previous experience at the National Institute of Administration in using a Moodle platform for knowledge sharing and evaluation and this was the main reason for choosing this solution. Also, in previous international projects implemented at INA, partners from similar public sector institutions France and Finland recommended the Moodle platform as a reliable solution for online training.
Moodle is one of the most popular learning management system (LMS). A learning management system (LMS) is a software application for the administration, documentation, tracking, reporting, automation and delivery of educational courses, training programs, or learning and development programs (Ellis, 2009). This type of platforms has two roles: on the one hand, they enable the content management (courses, homework), ensure synchronized collaboration (by chat, videoconferences), as well as non-synchronized collaboration (forum, messages, blog.) and, on the other hand, they can be used in managing the courses and the students that applied for these courses (Weller, 2007).
In LMS, mediation involves both the acquisition of competences and communication skills of all teachers and students, and a greater concern to create interaction moments and practical application possibilities of collaborative work, with that learning process happening in a participatory manner (Oliveira, 2015). Moodle can be successfully used both in the educational system, and in private or public institutions. Moodle can be used in universities in various purposes, both in full-time and part-time learning, e-learning or blended learning, both in the initial and continuous formation (Oproiu, 2015).
As Al-Ajlan & Zedan (2008) mentioned since 2008 more than 250 providers of commercial e-learning and more than 45 of them are Open Source Software (OSS) offerings as free VLE systems. We can say that the pandemic was the moment when the technologies were exploited to the maximum level and the opportunity expected by the software manufacturers. Latest market research reveals that the LMS market will reach a value of $28.1 billion by the end of 2025 (Imed, 2020). The same source reveals that LMS growth in Eastern Europe is at 16.9% annually.
Therefore, public institutions have the possibility to choose open-source software for online activities and Moodle could be a reliable solution. Wiliams (2005) pointed out that Moodle's low cost of implementation, flexibility and ease of use helps bring VLE technology within the reach of those with limited technical or financial resources.

Methodology
The current study focuses on aspects related to challenges, advantages, limitations and solutions in the process of transferring an entire training program to an online environment. In this purpose, we will describe the model used to make the transition from classical learning to online in the pilot project entitled PPP Online. By the end of the pilot project, 66 online training programs with 1252 participants were organized.
The main research objective is to validate a proposed model for organizing online training programs based on the analysis of the results of a pilot project and the participants' answers to the feedback questionnaires. This paper contains the proposed distribution of training hours by types of activities and the recommendations for trainers' for the asynchronous and synchronous activities.
In order to validate the proposed structure a feedback questionnaire was designed on Moodle. The questionnaire contains 12 questions structured in 3 sections. The first section aims to assess the participants' perception regarding the utility of the training program and the proposed plan of learning and evaluation. The second section focus on trainers' performance in teaching exclusively online. The last section aims to identify technical difficulties encountered and to highlight the experience of the participants during the program, including the interaction with trainers and other users.
We analysed responses to an online questionnaire survey that was administered between 18 September and 15 December 2020 to all participants from the online training programs organized by the Institute. A total of 1083 participants answered the questionnaire. All respondents are Romanian civil servants and 295 (27%) are from the central public administration institutions and ministries and 788 (73%) are from local public administration institutions. Also, we must mention that 340 (31%) of respondents are from Bucharest and 743 (69%) are from other 40 districts of Romania. The following table describes the demographic data on respondents:  Oliveira (2015) pointed out that with the e-learning advance, educational institutions managers started to deal with different activities, requiring the development of new procedures and finding alternatives to address emerging challenges that go beyond educational issues.

Results and discussions
Halverson (2019) mentioned that regardless of the complexity of the learning task, some students may find non-traditional settings like blended or online instruction to produce anxiety. In a recent study, Tartavulea (2020) stated that communication between students, between students and professors, immediate feedback, active learning and respecting differences between students in terms of skills and learning styles are perceived to have decreased with online teaching.
The proposed model for online teaching in the training programs conducted by the National Institute of Administration tried to limit the disadvantages caused by the lack of physical presence through a new approach and schedule.
At INA level, the decision was made to start a pilot project to facilitate the access for central and local public administration staff to online training programs, with free access. By this project INA aimed to: -increase its capacity to organize and develop online training programs (PPP online) in the future, starting with training the trainers and its staff for this process; -familiarize its participants with the online training to achieve professional development; -promote its services to the leaders of public authorities and institutions who will be able to evaluate the benefits of online training, with integration into the current program of participants, without creating blockages and / or interruptions in current work; -develop and refine an efficient, useful and practical concept for online professional development, by analysing and integrating lessons learned and feedback collected.
The characteristics of the online training programs, organised on INA's training platform are: -Total duration: 18 hours of training distributed in synchronous and asynchronous mode; -The synchronous mode: 8 hours distributed over several days, but not more than 2 hours / day of online presence on INA training platform (audio and video) simultaneously of both the trainer and the participants. The trainers will make interactive presentations based on the training materials previously published on INA platform. The activities carried out in synchronous mode can be recorded on the training platform and they can be accessed later by the participants during the training program. -The asynchronous mode: 10 hours, in which participants, individually, depending on the time available, will connect to INA's training platform and will respond to the requests made by the trainer through case studies, problems to be solved or other training materials. Practical applications are recommended and they can be solved through the online collaboration of the participants (including forums on various topics) through which the participants will learn from each other (not only from the trainer). The trainer will connect to the platform at different times from those of the participants, will respond to their questions, will organize discussion forums on certain topics and will evaluate the results obtained during each participant. -Duration of a training program: the 18 hours of training will be distributed in a maximum period of 2 weeks, according to a training schedule established by INA in collaboration with the trainer; -Number of participants: minimum 10 participants -maximum 20 participants; -Competencies evaluation is performed through a grid test (5 points), the evaluation of the results obtained along the way (4 points, 1 point for each practical application) by each participant, to which is added 1 point ex officio); -Certification of graduating the training programs is made by issuing Graduation Certificates, accompanied by the descriptive supplements regarding the acquired competencies, signed by INA's president.  By introducing the new Agenda with the distribution of hours and topics in synchronous and asynchronous format, we created the framework for: 1. Effective development of the training programs, in accordance with the principles of adult learning and incorporating good practices specific to the online training activity; 2. Facilitating the presence of participants to online training programs, during the daily work schedule and performing their duties; 3. Ensuring the traceability of professional training activities (theoretical and practical activity) and in compliance with the duration which is mentioned in the graduation certificate; 4. Application and compliance with quality standards specific to adult education and online training; 5. Compliance with the 1/3 ratio between theoretical training and practical training, according to the recommendations mentioned in the GD. no. 650/2016 for the approval of the Strategy for public administration training 2016-2020; 6. Certification of participation in the training program and / or acquisition of skills.
INA has constantly provided technical support for the smooth running of the professional development program.
The challenges of this way of supporting distance learning programs are: 1. Organizing the content of training materials to cover the 18 hours of training in synchronous and asynchronous format; 2. Design and balanced sizing of practical applications (exercises, case studies, etc.) so as to cover the time allocated to 90-minute sessions; 3. The correct evaluation of the results of the participants obtained in the realization of the practical applications and the transmission of the recommendations for their improvement; 4. Maintain a constant focus on participants and a high level of involvement. The scheduling of asynchronous activities is established as a benchmark to monitor the traceability of training hours and compliance with the work schedule of civil servants. According to GD no. 1.066 / 2008 for the approval of the norms regarding the professional training of civil servants Art. 42 and Art. 41 lit. c) civil servants participating in training programs have the following specific rights: to be allowed to participate in training programs, in accordance with the law, to have their achieved competencies recognised. According to Muñoz (2008) The identification of the weakest points of LMS platforms will depend on the objectives or requirements which are intended to fulfil and, therefore, a research for helping to choose the most adequate option should be made.
The LMS platform on which the PPP Online pilot project was developed was version 3.7 of Moodle and the Zoom application was used for synchronous activities. The major advantage of the Moodle platform is the customization facilities and the varied set of plugins that can be implemented. Initially, an attempt was made to integrate synchronous video sessions into the platform using other plugins, but it was concluded that, for a large number of users, the overall quality was not ensured.
By customizing the Moodle level of roles on the platform, the necessary access rights have been allocated for program managers, trainers and participants. A special role was also defined for training applicants before they were accepted as participants. The Moodle platform ensured the following flows: 1. Registration for training programs (including uploading the necessary documents to confirm registration). 2. Sharing learning resources by trainers. 3. Communication between program managers, participants and trainers (through messages, forum, chat). 4. Sending assignments to students and checking solutions by trainers. 5. Evaluation of the participants' competencies at the end of each training programs. 6. Automatic reporting of synthesis situations regarding the organization of programs. 7. Automatic issuance and transmission of graduation certificates in digital format, electronically signed. In this case, we can say that Moodle is a system that allows the management of training programs and participants throughout the training process (Figure 1). During the PPP online implementation, some customizations, but also developments were required. For the automatic sending of digitally signed certificates and the design of complex reports, the project team from INA made additional developments to the existing facilities on Moodle.
It should be mentioned that 22 trainers and 1252 participants were involved in the 66 training programs conducted online in the PPP online pilot project. As specified in the previous section, feedback questionnaires were collected from 1083 participants at the end of each online training program. The questionnaires included 12 questions and aimed to collect participants' feedback regarding the quality of online training and the difficulties encountered during the training process.
Perhaps the most important answer for organizers was at the question about the usefulness of the online training programs. From the received questionnaires, 979 respondents (82.8%) appreciated participation in the online training being very useful and 189 (16%) found the online training program useful. Regarding the proposed structure of the program, as can be seen from Table 6, more than 3 quarters of participants appreciated as very good the proportion between synchronous and asynchronous activities. The data in Table 7 illustrate a positive (good) perception from the participants regarding the online evaluation methods used in the programs (also more than 3 quarters appreciate very good). A synthesis of participants' perceptions of the pilot project is summarized in Figure 2: Another possible issue that was followed in the study was the ability of trainers to adapt to online training methods. As Bhati (2009) explained, teachers also face internal barriers including computer anxiety or fear of technology.
The 22 trainers were selected from the Institute's database and had no or little previous experience in online. Thus, in May 2020, the Institute organized a training session for trainers. During this session, 22 trainers explored the facilities of the Moodle platform and the Zoom platform. The configuration of the Moodle platform according to the specific training requirements was made by IT specialists from the Institute.
The results, as can be seen in Table 8, were very good regarding the trainers performances with more than 90% positive responses. The study also aimed to observe how participants adapted to online training. There have been concerns about the difficulty of using new technologies needed for online training. The study reveals that approximately 56% of the participants considered the degree of demand higher or very high compared to the classic training programs, while 44% did not have problems in the online environment. As expected, there were some technical problems, most of which were solved shortly by the team. Respondents were asked if they had any technical problems and to detail them. The study tracked potential difficulties in connection with: registration on the Moodle platform, accessing their account, communication on the forum and chat, communication with trainers through Moodle's message system, technical problems on the Zoom platform and problems related to internet connection.
The study revealed that 842 (77%) participants did not encounter problems accessing online technologies. However, 241 participants mentioned one or more difficulties, mostly with internet connection (187) and video or audio access to the Zoom platform (172). We can appreciate that these connectivity issues are due in most cases to the hardware of the participants. The assessment of the specialized and continuous training programs provided by INA in 2018 and 2019, based on a detailed satisfaction questionnaire sent to 4474 participants, with 710 answers (15.8% response rate) revealed that, in the case of face to face programs, the majority of participants (92%) considered the training programs to be useful (39%) and very useful (53%). From this perspective, it can be appreciated that performances the level of performance of online programs is comparable to that previously obtained (INA, 2020).

Conclusion
At National Institute of Administration, through the PPP Online pilot project, a 100% online training process was developed, tested and refined with the optimal distribution of learning time and integration of training in the work schedule. A methodology was designed and applied for the entire training process, including the delivery of digital graduation certificates with the registration of acquired competencies and security elements. A body of trainers and also the INA training program managers were trained to design and implement online programs. From the experience exchange with other international training institutions, this approach was used before starting the implementation of online training programs. Within the pilot project, quality standards, procedures and instructions have been developed and applied for managing the relationship with participants, content development, training and certification activities to ensure an optimal level of transparency and traceability of the services provided.
The proposed structured and the distribution of training hours have been validated by the positive feedback from the participants attending the training programs organized in the pilot program. More than 80% of the participants had very favourable reviews regarding the usefulness, structure and online evaluation methods used. We can appreciate that trainers, despite of having little previous experience in online training, have adapted remarkably in a short time to the online teaching environment.
Attending online programs has been considered more demanding than attending classical programs (face-to-face) by about 56% of participants, but it is a normal aspect related to the lack of previous experiences. Technical problems are not an obstacle in the light of a good configuration of software platforms and many of them will be overcome once the participants will get accustomed with digital technologies and instruments. The graduation rate for online training programs was 93%, similar to that of programs in classical format. Also, the distribution of participants' grades was similar.
The limitations of the research are related to the context of the pandemic, especially to the fact that the participants could not benefit from another form of training. Also, the trainers involved in the project carried out the activity on a voluntary basis and had a limited time to adapt the materials for online training. Due to the fact that participants within the pilot project did not pay any participation fee, it is possible that the level of demand will be lower than normal and could influence the results of the analysis based on the questionnaire.
We appreciate that the research can be extended over a longer period of time in order to remove the limitations imposed by the pandemic and to evaluate the options of public sector participants between online training programs or the traditional system. The deficiencies reported by the participants in the classical training programs and the advantages of online training could be analysed.
We can conclude that the advantages obtained can be used not only in the context of the pandemic, but also for the organizing future training programs for the public sector.