1. bookVolume 3 (2018): Issue 4 (November 2018)
Journal Details
License
Format
Journal
First Published
30 Mar 2017
Publication timeframe
4 times per year
Languages
English
access type Open Access

Applications of, and Experiences with, the Norwegian Model in Finland

Published Online: 07 Dec 2018
Page range: 31 - 44
Received: 03 Sep 2018
Accepted: 25 Oct 2018
Journal Details
License
Format
Journal
First Published
30 Mar 2017
Publication timeframe
4 times per year
Languages
English

The purpose of this article is to describe the development, components and properties of a publication indicator that the Ministry of Education and Culture in Finland uses for allocating direct core funding annually to universities. Since 2013, 13% of the core funding has been allocated on basis of publication indicator that, like the Norwegian model, is based on comprehensive national level publication data that is currently provided by the VIRTA publication information service. In 2015, the publication indicator was complemented with other components of the Norwegian model, namely, quality-weighted publication counts based on national Publication Forum authority list of the publication channels with ratings established by experts in the field. The funding model allocates around 1.6 billion euros annually to universities with the publication indicator annually distributing over 200 million euros. Besides the funding model, the indicator provides comparable data for monitoring the research performance of Finnish universities, fields and subunits. The indicator may also be used in the universities’ local funding models and research management systems, sometimes even at individual level evaluation. Positive and negative effects of the indicator have been extensively discussed and speculated. Since 2011, the Finnish universities’ productivity appears to have increased in terms of both quantity and quality of publications.

Keywords

Background and motivation

University funding by government in Finland has been partly based on performance since the establishment in the 1990s of performance agreements, funding models and the KOTA-database for input and output statistics. The main policy goals have been to enhance efficiency, internationalisation, quality and impact of educational and research activities, gaining insights into the performance of the Finnish higher education institutions, as well as creating accountability and transparency regarding the use of public funds. Adaptation of the Norwegian model is embedded in the funding model that the Ministry of Education and Culture uses for allocating direct core funding annually to universities.

The funding model determines what performance indicators are used, and what is their share of core funding, in a fixed funding formula. The next years’ funding is calculated on the basis of three previous year’s performance outputs (average). Over the years, performance-based funding criteria have undergone several changes. In general, the share of core funding allocated on basis of indicators, and the weight of research indicators compared to education, has been increased. Publication statistics have been collected since 1994. The number of scientific publications related to the number of academic staff was introduced to the model in 2007–2012 as an indicator of research performance.

In the current model used since 2017, 41% of core funding is allocated on basis of indicators measuring performance in education, 33% is based on research performance indicators, and 28% on other policy considerations (Table 1). One of four research indicators is the quality-weighted count of scientific publications (share 13%). This indicator was adapted with components of the Norwegian model gradually in 2013–2015.

Universities’ funding model 2017–2020.

Performance indicatorsShare (%)
Education41

Master’s degrees

13

Bachelors’ degrees

6

Study credits in open university, specialization studies, studies based on cooperation and in non-degree programmes

2

Number of students who have gained at least 55 study credits

10

Student feedback

3

Number of employed graduates

2

Master’s degrees awarded to foreign nationals

1

Student mobility to and from Finland

2
Research33

PhD degrees

9

Scientific publications

13

International teaching and research staff

2

Competed research funding

9
Other education and science policy objectives28

Strategic development

12

Field-specific funding

9

National duties

7
Organization of (re)design, implementation and operations

Development of a publication-based indicator for measuring research performance was discussed and planned since 2005 in several ministry working-groups and reports that investigated publication practices, information sources and international examples. In 2007, a KOTA working-group reviewed the performance-based research funding systems (PRFSs) used at the time in Australia, Flanders, Norway, Sweden and UK. Following the Norwegian model, the KOTA working-group recommended the development of a comprehensive national publication information system to improve the quality of publication data, as well as the development of a national authority list of publication channels to take quality of publications adequately into account in the funding model.

In 2009, the Finnish Council of University Rectors (now Universities Finland UNIFI) set up a working-group to develop the evaluation of quality of publications. Citation analysis, which is well established in science and medicine, was not chosen because of the inadequate coverage of the social sciences and humanities publications in the international databases (Web of Science and Scopus). Based on the Norwegian model adopted already in Denmark, also the UNIFI working-group recommended the development in Finland of a system based on a publication forum, in which peer-reviewed publication channels are identified and classified in quality levels by the experts in the field. Several advantages and disadvantages were identified with this model (Table 2).

UNIFI working-group’s SWOT analysis of a publication forum-based system.

StrengthsWeaknesses

promotes quality

burdensome expert-panel organisation

quality defined by research community

panels tie human resources

sensitive to different publishing practices

requires harmonization of publication data

could be established e.g. only for SSH fields

renewal of field-classification (in Finland)

transparent indication of quality

validity may differ between fields

takes into account quantity and quality

comprehensive information for all fields

dynamic system that can be updated

OpportunitiesThreats

current data about research quality

experts privilege channels of national interest

supports research assessments

over quality in ratings

steering and management

ensuring commensurability

field and subfield specific analyses

difficult to reach consensus about ratings

Data sources and indicator design

There is a clear distinction of responsibilities with regard to the design of the publication indicator and the development of its different components: Federation of Finnish Learned Societies (TSV) produces together with the research community the publication channel authority list with level ratings, CSC – IT Center for Science Ltd. collects the comprehensive national level publication data from universities, while the funding model working-groups constituted by the Ministry Education and Culture decide if and how the publication data and channel ratings are used in the funding model indicator.

TSV, an independent organisation that represents the research community via its over 250 member societies, has been entrusted with the responsibility to produce the national authority list of peer-reviewed publication channels. Funding for this and other TSV activities comes from the Ministry. In 2010, TSV appointed a steering-group for a two-year project, which established a secretariat (2 persons fulltime) and appointed 23 expert panels covering all fields specifically for the task of evaluating the outlets. Panels have around 250 members from Finnish universities and public research institutes. The first authority list was published in the beginning of 2012, and after the project ended, the Publication Forum has become a fixed part of the TSV activities.

The most notable differences between the Norwegian and the Finnish authority lists concern the number of panels, the number of quality levels, the rating of national language channels, and the interval of updates. The rating has three levels for peer-reviewed publication series and book publishers: level 3 = top, level 2 = leading and level 1 = basic. There is also a level 0 for channels not qualifying as level 1. Both foreign and national journals/series, conferences and book publishers can be admitted to level 1 if they meet the following four criteria:

specialised in the publication of scientific or scholarly research outcomes

editorial board constituted by experts

entire manuscripts of scientific or scholarly articles or books subject to peer review

registered ISSN or ISBN number

As a main rule, however, even a publication channel meeting these criteria should not be included in Level 1 if they are local (mainly used by researchers of a single research organisation) or the quality and relevance to Finnish research community is questionable (e.g. predatory journals).

The authority list contains more than 30 000 publication series and book publishers, of which levels 2 and 3 include 10% of the most widely respected international journals and book publishers in different fields (table 3). In the social sciences and humanities fields, however, level 2 has also included since 2012 three book publishers and over 20 journals and book series publishing in the national languages (Finnish and Swedish). To balance the ratings across fields, in each expert-panel the publication channels selected for level 2 can represent in total up to 15%, and in level 3 up to 5%, of the world’s publications.

Publication Forum authority list of publication channels in 2018.

Publication Forum levelJournals/seriesBook publishers
365415
22,24291
119,8371,203
05,0471,710
All27,7803,019

The authority list of publication channels is updated regularly, new channels being evaluated for addition to level 1 every year, and levels 2 and 3 are updated every four years. Also, the panels and the steering-group are appointed for four-year periods. Publication Forum ratings are openly available at the JUFO-portal, which the members of research community can also use for suggesting new additions and changes to the ratings.

CSC, a company partly owned by the state of Finland (70%) and Finnish higher education institutions (30%), has developed technical solutions for the national level integration of publication metadata from the universities since 2009. Finland faced the challenge that many universities had strongly invested in their own information systems, so it was not considered feasible to replace those systems with a national CRIS (such as CRISTIN in Norway). Integrating data from diverse local information systems has involved development of the data model, the publication type and field classifications, as well as the data collection guidelines and procedures to ensure quality and comparability of the publication data. Full publication data from 2011 to the present is currently stored in the VIRTA publication information service, an advanced solution launched in 2016.

In VIRTA, publications are divided in scientific, professional and general publications, and for the scientific publications separate publication types are used for the refereed and not-refereed articles in journals, conferences and books, monographs and edited work (Table 4). Publications are assigned to the publication types in universities at the time of registering of publications in the local information systems, from which these classifications are transferred with publication metadata to VIRTA. VIRTA stores records also for artistic and design activities, theses, patents and innovation announcements, as well as audiovisual material and ICT applications, but these have not been taken into account in the publication indicator.

Publication type classification in VIRTA publication information service.

Scientific publications
A1 Peer-reviewed journal article, original research
A2 Peer-reviewed journal article, review
A3 Peer-reviewed article or chapter in book
A4 Peer-reviewed article in conference proceedings
B1 Non-peer-reviewed journal article
B2 Non-peer-reviewed article or chapter in book
B3 Non-peer-reviewed article in conference proceedings
C1 Monograph (peer-reviewed)
C2 Edited book or special issue (peer-reviewed)
Professional publications
D1 Article in a trade journal
D2 Article in a professional book
D3 Article in professional conference proceedings
D4 Published development or research report or study
D5 Textbook, professional manual or guide
D6 Edited professional book
Publications for the general public
E1 Popularised article, newspaper article
E2 Popularised monograph
E3 Edited popularised book

According to the data collection guidelines published by the Ministry of Education and Culture, scientific publications need to fulfil the following three conditions:

The publication must produce new information in relation to previous research data on the same subject.

The publication must be presented in a format that enables the verification of the research results and/or use of the research results in a new research, thus allowing other researchers to assess the research results and use them in their own work.

The publication channel for the publication specialises in publishing scientific research results, and it has an editorial staff consisting of experts in the field of science as well as a peer review practice.

A pre-publication peer-review, focusing on research quality, by independent experts in the field is required for inclusion of publications among the peer-reviewed publication types (A and C). To support the identification of peer-reviewed publications, TSV introduced in 2014 a national label for peer-reviewed publications inspired by the example of the GPRC-label in Flanders. Ten Finnish book publishers and 167 journals currently use the label to indicate articles and books that have undergone peer-review according to the label requirements.

The funding model is developed by working-groups appointed periodically by the Ministry to consider adjustments to the performance indicators and their share of the core funding. These groups have representatives from the Ministry, the higher education institutions, and the Academy of Finland (the main research funding agency). A funding model proposal is published in a report and sent out by the Ministry for consultation to a wide group of stakeholders. A decree concerning the universities’ core funding criteria is amended according to the new model approved by the Ministry.

The publication information now stored in VIRTA has been used as the data source of publication indicator since 2013. Weighting of publications according to the publication type was also introduced already in 2013, as monographs (C1) were counted with four times larger weight than articles (A1–A4, B 1–B3). The Publication Forum levels have been identified for all peer-reviewed articles and monographs published in 2011 or later. In the period 2013–2014, however, the funding model did not yet use the Publication Forum levels to differentiate publications according to quality. Instead, 9% of core funding was allocated on basis of an undifferentiated count of international refereed publications and 4% based on the other publications. The former category included publication types A1–A4 and C1 with country of publication other than Finland, and the latter category included all the other publications belonging to types A, B and C.

The funding model working-group originally planned that in 2015, international refereed publications would be replaced with the count of publications in Publication Forum level 2 and 3 channels, and other publications with publications in level 1 channels. Eventually, a weighting of publications by type and channel was introduced in 2015, and later adjusted for the period 2017–2020 (Tables 5 and 6). The Finnish publication indicator differs somewhat from the Norwegian model as it assigns similar weight to peer-reviewed articles in journals, books and proceedings, and takes into account with small weight not-refereed publications intended for scholarly, professional and general audiences (publication types D and E).

Weight of publications in the funding model 2015–2016.

Publication typeLevel 3Level 2Level 1Level 0
Peer-reviewed monograph (C1)121264
Peer-reviewed article in journal (A1–2)331.51
Peer-reviewed article in book (A3)331.51
Peer-reviewed article in proceedings (A4)331.51
Not-peer-reviewed monographs0.4
Not-peer-reviewed articles0.1

Weight of publications in the funding model since 2017–2020.

Publication typeLevel 3Level 2Level 1Level 0
Peer-reviewed monograph (C1)161240.4
Peer-reviewed article in journal (A1–2)4310.1
Peer-reviewed article in book (A3)4310.1
Peer-reviewed article in proceedings (A4)4310.1
Peer-reviewed edited work (C2)4310.1
Not-peer-reviewed monographs0.4
Not-peer-reviewed articles and edited work0.1

Institutional level whole-counts are used, meaning that co-publications with authors from several Finnish universities are counted more than once in the funding model. Fractionalization at institutional or author level has been discussed but not adopted in order not to discourage inter-university collaboration.

The Universities Act and the Polytechnics Act require all higher-education institutions, including 14 universities and 23 universities of applied sciences, to supply publication information to Ministry of Education and Culture. Also five university hospital districts, each consisting of several hospitals, and six public research institutes have agreed to provide their publication data to VIRTA. The publication information is openly available at the JUULI-portal for browsing and searching, and at the VIPUNEN-portal for statistics.

Funding implications

The funding model allocates around 1.6 billion euros annually to universities, and this core funding covers almost two-thirds of the universities’ budgets. The share of core funding allocated on basis of scientific publications has increased considerably, being 0.3% in 2007–2009, 1.7% in 2010–2012, and 13% since 2013. Since 2013, the publication indicator distributes annually over 200 million euros. In a report published in 2018 by Economic Policy Council on the university reform and financial Incentives, it was estimated that one publication point (equivalent e.g. to one peer-reviewed article in level 1 journal) is worth approximately 4200 euros to universities (Seuri & Vartiainen, 2018).

Other uses of the data

According to the UNIFI working-group, the development of the Publication Forum based publication indicator supported with national level publication data would serve three potential interests:

to provide quality indicator of publications for the universities’ funding model

to provide universities with comparable data for benchmarking with other universities, monitoring the publication profiles of units, and developing internal funding models

to increase the awareness of the researchers about quality of publication channels, thereby raising their level of ambition in publishing

The working-group also pointed out that Publication Forum could be used as information supporting research evaluation, for example in research assessment exercises the universities conduct themselves every few years, and the state of research in Finland reports produced by the Academy of Finland. Publication Forum ratings have indeed been used to supplement Web of Science based bibliometric analyses in some recent research assessment exercises.

Besides the funding model, the Ministry of Education and Culture uses the information based on the VIRTA publication data and the Publication Forum ratings for monitoring the research performance of the universities. Since 2010, the Ministry’s spending proposals submitted to the Ministry of Finance for the formulation of the state budget includes quantitative research performance targets for the universities collectively. Since 2016, targets have been set for the number of scientific level 2 and 3 publications per academic staff, as well as the international co-publications’ share of scientific level 1–3 publications. The same indicators are used by the Ministry as positioning statistics, openly available at the VIPUNEN-portal, for the performance agreements with individual universities.

The Ministry of Education and Culture has emphasized that the internal allocation of funds to faculties and departments should be based on universities’ own strategic priorities rather than replicate the funding model criteria. Nevertheless, universities are autonomous in using data and indicators in evaluation, development and management of their research activities. While the UNIFI working-group envisioned the potential use of the indicator in universities, appropriate local models and uses of the Publication Forum rating have not been defined. Already in 2012, the Publication Forum steering-group published a user-guide, in which it recommended not to use the rating in cross-field comparisons or in the evaluation of individual researchers.

In the fall 2015, TSV and UNIFI conducted a survey among university managers concerning the local use of the Publication Forum rating. The sample consisted of 10 rectors, 19 deans and 68 heads of department. Most Finnish universities make use of the channel rating for monitoring and developing their publishing activities, and in some universities the rating is used also for funding allocation to faculties and/or departments. The use of the ratings for evaluation purposes (recruitment, promotion, personal performance, bonuses) at the individual level is also attested, especially in social sciences and humanities. It is not clear, however, to what extent the rating supports, rather than supplants, qualitative expert assessment.

VIRTA publication data, including the Publication Forum ratings, has provided the source of information for bibliometric research and analyses concerning for example publications patterns and research productivity.

Experiences and effects

In Finland, as in the other Nordic countries using the Norwegian model, the positive and negative effects of the publication indicator have been discussed and speculated extensively. The effect of the publication indicator on research performance has been investigated in two recent reports, one evaluating the impact of the Universities Act reform in 2010 (2016), and another by Economic Policy Council assessing specifically the incentives created by the funding model (2018). In both reports, the development of Finnish universities publication activities was analyzed using the VIRTA publication data and Publication Forum levels. Both reports conclude that the research productivity has increased since 2011. Most notably, there has been a clear movement from level 0 channels toward level 1–3 channels. It seems that the publication channel rating has increased the Finnish research community’s awareness of quality of publication channels, even though the quality-weights were formally introduced to the funding model only in 2015.

The Economic Policy Council’s report also established that the average Publication Forum rating of peer-reviewed publications has increased in all universities, and more so in those universities where the average level was weaker in 2011. The differences in performance have decreased between universities, suggesting that the model has been most effective in those environments where full potential in productivity have not yet been reached. When related to the number of teaching and research personnel, it seems that the publication productivity of academic staff has increased but this has not happened at the expense of quality. But also the question is raised, to what extent the level ratings indicate increase in quality, as the selection of channels to level 2 and 3 can be influenced by local interests. The report also pointed out field specific differences in publication performance, making some fields seem more productive to universities.

Although it is regarded an advantage of the publication indicator that research community itself is entrusted the task of rating the publication channels, subjectivity of the expert-ratings is a common concern in Finland. The indicator has been developed for the purpose of providing comprehensive coverage of research outputs across fields, but equal treatment of different fields and specialties in level 2 and 3 nominations is much debated. The indicator is sensitive to the specificity of social sciences and humanities publishing culture, yet the funding models’ incentives are more often linked with decline rather than promotion of book and national language publishing. Other topics of discussion have included the effect of the publication indicator on interdisciplinarity and open access. Many of the concerns probably stem from the the publication indicator’s use for research management and evaluation purposes at universities, rather than from the indicators’ use in the universities’ funding model.

Publication type classification in VIRTA publication information service.

Scientific publications
A1 Peer-reviewed journal article, original research
A2 Peer-reviewed journal article, review
A3 Peer-reviewed article or chapter in book
A4 Peer-reviewed article in conference proceedings
B1 Non-peer-reviewed journal article
B2 Non-peer-reviewed article or chapter in book
B3 Non-peer-reviewed article in conference proceedings
C1 Monograph (peer-reviewed)
C2 Edited book or special issue (peer-reviewed)
Professional publications
D1 Article in a trade journal
D2 Article in a professional book
D3 Article in professional conference proceedings
D4 Published development or research report or study
D5 Textbook, professional manual or guide
D6 Edited professional book
Publications for the general public
E1 Popularised article, newspaper article
E2 Popularised monograph
E3 Edited popularised book

Universities’ funding model 2017–2020.

Performance indicatorsShare (%)
Education41

Master’s degrees

13

Bachelors’ degrees

6

Study credits in open university, specialization studies, studies based on cooperation and in non-degree programmes

2

Number of students who have gained at least 55 study credits

10

Student feedback

3

Number of employed graduates

2

Master’s degrees awarded to foreign nationals

1

Student mobility to and from Finland

2
Research33

PhD degrees

9

Scientific publications

13

International teaching and research staff

2

Competed research funding

9
Other education and science policy objectives28

Strategic development

12

Field-specific funding

9

National duties

7

Publication Forum authority list of publication channels in 2018.

Publication Forum levelJournals/seriesBook publishers
365415
22,24291
119,8371,203
05,0471,710
All27,7803,019

Weight of publications in the funding model since 2017–2020.

Publication typeLevel 3Level 2Level 1Level 0
Peer-reviewed monograph (C1)161240.4
Peer-reviewed article in journal (A1–2)4310.1
Peer-reviewed article in book (A3)4310.1
Peer-reviewed article in proceedings (A4)4310.1
Peer-reviewed edited work (C2)4310.1
Not-peer-reviewed monographs0.4
Not-peer-reviewed articles and edited work0.1

Weight of publications in the funding model 2015–2016.

Publication typeLevel 3Level 2Level 1Level 0
Peer-reviewed monograph (C1)121264
Peer-reviewed article in journal (A1–2)331.51
Peer-reviewed article in book (A3)331.51
Peer-reviewed article in proceedings (A4)331.51
Not-peer-reviewed monographs0.4
Not-peer-reviewed articles0.1

UNIFI working-group’s SWOT analysis of a publication forum-based system.

StrengthsWeaknesses

promotes quality

burdensome expert-panel organisation

quality defined by research community

panels tie human resources

sensitive to different publishing practices

requires harmonization of publication data

could be established e.g. only for SSH fields

renewal of field-classification (in Finland)

transparent indication of quality

validity may differ between fields

takes into account quantity and quality

comprehensive information for all fields

dynamic system that can be updated

OpportunitiesThreats

current data about research quality

experts privilege channels of national interest

supports research assessments

over quality in ratings

steering and management

ensuring commensurability

field and subfield specific analyses

difficult to reach consensus about ratings

Description of the Finnish higher education system, performance agreements and funding model in EnglishDescription of the Finnish higher education system, performance agreements and funding model in EnglishSearch in Google Scholar

Aarrevaara, T., Wahlfors, L., & Dobson, I. (2018). Higher Education Systems and Institutions, Finland. In Teixeira, Pedro et al. (eds.), Encyclopedia of International Higher Education Systems and Institutions, Springer 2018.AarrevaaraT.WahlforsL.DobsonI.2018Higher Education Systems and Institutions, FinlandTeixeiraPedroEncyclopedia of International Higher Education Systems and InstitutionsSpringer2018Search in Google Scholar

De Boer, H.F., Jongbloed, B., Benneworth, P., Cremonini, L., Kolster, R., Kottmann, A., Lemmens-Krug, K., & Vossensteyn, H. (2015). Performance-based funding and performance agreements in fourteen higher education systems—Report for the Ministry of Education, Culture and Science, Enschede: CHEPS, University of Twente. Retrieved from https://ris.utwente.nl/ws/portalfiles/portal/5139542/jongbloed+ea+performance-based-funding-and-performance-agreements-in-fourteen-higher-education-systems.pdfDe BoerH.F.JongbloedB.BenneworthP.CremoniniL.KolsterR.KottmannA.Lemmens-KrugK.VossensteynH.2015Performance-based funding and performance agreements in fourteen higher education systems—Report for the Ministry of Education, Culture and ScienceEnschedeCHEPS, University of TwenteRetrieved fromhttps://ris.utwente.nl/ws/portalfiles/portal/5139542/jongbloed+ea+performance-based-funding-and-performance-agreements-in-fourteen-higher-education-systems.pdfSearch in Google Scholar

Jonkers, K., & Zacharewicz, T. (2016). Research Performance Based Funding Systems: a Comparative Assessment. European Union. Retrieved from http://publications.jrc.ec.europa.eu/repository/bitstream/JRC101043/kj1a27837enn.pdfJonkersK.ZacharewiczT.2016Research Performance Based Funding Systems: a Comparative AssessmentEuropean UnionRetrieved fromhttp://publications.jrc.ec.europa.eu/repository/bitstream/JRC101043/kj1a27837enn.pdfSearch in Google Scholar

Reports related to the development of the funding model and the publication indicatorReports related to the development of the funding model and the publication indicatorSearch in Google Scholar

Ministry of Education (2005). Yliopistojen tulosohjauksen kehittämistyöryhmä III (in English: Committee on the development of university performance management III). Helsinki: Opetusministeriö. Retrieved from https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/80321/tr24.pdf?sequence=1Ministry of Education2005Yliopistojen tulosohjauksen kehittämistyöryhmä III(in English: Committee on the development of university performance management III)HelsinkiOpetusministeriöRetrieved fromhttps://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/80321/tr24.pdf?sequence=1Search in Google Scholar

Ministry of Education (2007). Bibliometristen aineistojen käytettävyys yliopistojen julkaisujen laadun ja tuottavuuden arvioinnissa (in English: Usability of bibliometric data in evaluation of quality and productivity of universities’ publications). Helsinki: OPetusministeriö. Retrieved from http://www.minedu.fi/export/sites/default/OPM/Julkaisut/2007/liitteet/opm02.pdf?lang=fiMinistry of Education2007Bibliometristen aineistojen käytettävyys yliopistojen julkaisujen laadun ja tuottavuuden arvioinnissa(in English: Usability of bibliometric data in evaluation of quality and productivity of universities’ publications)HelsinkiOPetusministeriöRetrieved fromhttp://www.minedu.fi/export/sites/default/OPM/Julkaisut/2007/liitteet/opm02.pdf?lang=fiSearch in Google Scholar

Ministry of Education (2008). Julkaisukäytännöt eri tieteenaloilla (in English: Publishing patterns in different fields). Helsinki: Opetusministeriö. Retrieved from http://www.minedu.fi/export/sites/default/OPM/Julkaisut/2008/liitteet/opm33.pdf?lang=fi;Ministry of Education2008Julkaisukäytännöt eri tieteenaloilla(in English: Publishing patterns in different fields)HelsinkiOpetusministeriöRetrieved fromhttp://www.minedu.fi/export/sites/default/OPM/Julkaisut/2008/liitteet/opm33.pdf?lang=fi;Search in Google Scholar

Ministry of Education (2008). TiedeKOTA-työryhmän raportti (In English: Report of the TiedeKOTA working-group). Helsinki: Opetusministeriö. Retrieved from http://docplayer.fi/14481463-Tiedekota-tyoryhman-raportti.htmlMinistry of Education2008TiedeKOTA-työryhmän raportti(In English: Report of the TiedeKOTA working-group)HelsinkiOpetusministeriöRetrieved fromhttp://docplayer.fi/14481463-Tiedekota-tyoryhman-raportti.htmlSearch in Google Scholar

Ministry of Education and Culture (2011). Laadukas, kansainvälinen, profiloitunut ja vaikuttava yliopisto – ehdotus yliopistojen rahoitusmalliksi vuodesta 2013 alkaen (In English: High-quality, profilised and effective international university - Proposal for a reform of the university financing model from 2013). Helsinki: Opetus- ja kulttuuriministeriö. Retrieved from http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75464/tr26.pdf?sequence=1&isAllowed=yMinistry of Education and Culture2011Laadukas, kansainvälinen, profiloitunut ja vaikuttava yliopisto – ehdotus yliopistojen rahoitusmalliksi vuodesta 2013 alkaen(In English: High-quality, profilised and effective international university - Proposal for a reform of the university financing model from 2013)HelsinkiOpetus- ja kulttuuriministeriöRetrieved fromhttp://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75464/tr26.pdf?sequence=1&isAllowed=ySearch in Google Scholar

Ministry of Education and Culture (2014). Vahvemmat kannusteet koulutuksen ja tutkimuksen laadun vahvistamiselle: ehdotus yliopistojen rahoitusmallin tarkistamiseksi vuodesta 2015 alkaen (In English: Greater incentives for strengthening quality in education and research: A proposal for revising the funding model for universities as of 2015). Helsinki: Opetusja kulttuuriministeriö. Retrieved from https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75264/tr07.pdf?sequence=1Ministry of Education and Culture2014Vahvemmat kannusteet koulutuksen ja tutkimuksen laadun vahvistamiselle: ehdotus yliopistojen rahoitusmallin tarkistamiseksi vuodesta 2015 alkaen(In English: Greater incentives for strengthening quality in education and research: A proposal for revising the funding model for universities as of 2015)HelsinkiOpetusja kulttuuriministeriöRetrieved fromhttps://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75264/tr07.pdf?sequence=1Search in Google Scholar

Ministry of Education and Culture (2015). Ehdotus yliopistojen rahoitusmalliksi 2017 alkaen (In English: Proposal for the funding model of universities as of 2017). Helsinki: Opetus- ja kulttuuriministeriö. Retrieved from https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75157/tr19.pdf?sequence=1Ministry of Education and Culture2015Ehdotus yliopistojen rahoitusmalliksi 2017 alkaen(In English: Proposal for the funding model of universities as of 2017)HelsinkiOpetus- ja kulttuuriministeriöRetrieved fromhttps://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75157/tr19.pdf?sequence=1Search in Google Scholar

UNIFI (2010). Julkaisujen laadunarvioinnin kehittäminen (In English: Development of evaluation of publication quality). Retrieved from http://www.helsinki.fi/halvi/srno/raportit_ja_julkaisut/Julkaisujen_laadunarviointi.pdfUNIFI2010Julkaisujen laadunarvioinnin kehittäminen(In English: Development of evaluation of publication quality). Retrieved fromhttp://www.helsinki.fi/halvi/srno/raportit_ja_julkaisut/Julkaisujen_laadunarviointi.pdfSearch in Google Scholar

Ministry of education and culture: Retrieved from https://minedu.fi/en/steering-financing-and-agreementsMinistry of education and culture: Retrieved fromhttps://minedu.fi/en/steering-financing-and-agreementsSearch in Google Scholar

Budget spending proposals submitted to the Ministry of Finance: Retrieved from http://budjetti.vm.fi/indox/index.jspBudget spending proposals submitted to the Ministry of Finance: Retrieved fromhttp://budjetti.vm.fi/indox/index.jspSearch in Google Scholar

Documents describing the development and properties of the publication channel ratingDocuments describing the development and properties of the publication channel ratingSearch in Google Scholar

Auranen, O., & Pölönen, J. (2012). Classification of scientific publication channels: Final report of the Publication Forum project (2010–2012). Helsinki: Federation of Finnish Learned Societies. Retrieved from http://www.julkaisufoorumi.fi/sites/julkaisufoorumi.fi/files/publication_forum_project_final_report_0.pdfAuranenO.PölönenJ.2012Classification of scientific publication channelsFinal report of the Publication Forum project(2010–2012)HelsinkiFederation of Finnish Learned SocietiesRetrieved fromhttp://www.julkaisufoorumi.fi/sites/julkaisufoorumi.fi/files/publication_forum_project_final_report_0.pdfSearch in Google Scholar

Pölönen, J., & Ruth, A. S. (2015). Final report on 2014 review of ratings in Publication Forum. Helsinki: Federation of Finnish Learned Societies. Retrieved from http://www.julkaisufoorumi.fi/sites/julkaisufoorumi.fi/files/publication_forum_final_report_on_2014_review_of_ratings.pdfPölönenJ.RuthA. S.2015Final report on 2014 review of ratings in Publication ForumHelsinkiFederation of Finnish Learned SocietiesRetrieved fromhttp://www.julkaisufoorumi.fi/sites/julkaisufoorumi.fi/files/publication_forum_final_report_on_2014_review_of_ratings.pdfSearch in Google Scholar

Publication Forum Manual for Panels (2018). Retrieved from https://wiki.eduuni.fi/display/CSCJulkaisufoorumiportaali/Publication+Forum+Manual+for+PanelsPublication Forum Manual for Panels2018Retrieved fromhttps://wiki.eduuni.fi/display/CSCJulkaisufoorumiportaali/Publication+Forum+Manual+for+PanelsSearch in Google Scholar

User-guide (2018). Retrieved from http://www.julkaisufoorumi.fi/sites/julkaisufoorumi.fi/files/jufo_manual_2018v1.pdfUser-guide2018Retrieved fromhttp://www.julkaisufoorumi.fi/sites/julkaisufoorumi.fi/files/jufo_manual_2018v1.pdfSearch in Google Scholar

Publication Forum website: www.julkaisufoorumi.fiPublication Forum websitewww.julkaisufoorumi.fiSearch in Google Scholar

JUFO-portal: Retrieved from http://jfp.csc.fi:8080/en/JUFO-portal: Retrieved fromhttp://jfp.csc.fi:8080/en/Search in Google Scholar

Label for peer-reviewed publications: Retrieved from https://www.tsv.fi/en/services/label-for-peer-reviewed-scholarly-publicationsLabel for peer-reviewed publications: Retrieved fromhttps://www.tsv.fi/en/services/label-for-peer-reviewed-scholarly-publicationsSearch in Google Scholar

Description of the properties of the national publication dataDescription of the properties of the national publication dataSearch in Google Scholar

Sile, L., Guns, R., Sivertsen, G., & Engels, T. C. E. (2017). European Databases and Repositories for Social Sciences and Humanities Research Output. Antwerp: ECOOM & ENRESSH. Retrieved from https://doi.org/10.6084/m9.figshare.5172322.v2SileL.GunsR.SivertsenG.EngelsT. C. E.2017European Databases and Repositories for Social Sciences and Humanities Research OutputAntwerpECOOM & ENRESSHRetrieved fromhttps://doi.org/10.6084/m9.figshare.5172322.v2Search in Google Scholar

Sile, L., Pölönen, J., Sivertsen, G., Guns, R., Engels, T., Arefievd, P., Duškováe, M., Faurbækf, L., Hollg, A., Kulczycki, E., Nelhans, G., Petr, M., Piskk, M., Soós, S., Stojanovskil, J., Stone, A., Šušolo, J., & Teitelbaump, R. (2018). Comprehensiveness of national bibliographic databases for social sciences and humanities: findings from a European survey, Research Evaluation doi: 10.1093/reseval/rvy016.SileL.PölönenJ.SivertsenG.GunsR.EngelsT.ArefievdP.DuškováeM.FaurbækfL.HollgA.KulczyckiE.NelhansG.PetrM.PiskkM.SoósS.StojanovskilJ.StoneA.ŠušoloJ.TeitelbaumpR.2018Comprehensiveness of national bibliographic databases for social sciences and humanities: findings from a European survey, Research Evaluation10.1093/reseval/rvy016Open DOISearch in Google Scholar

Puuska H. M., Guns, R., Pölönen, J., Sivertsen, G., Mañana-Rodríguez, J., & Engels, T.C.E. (2018). Proof of concept of a European database for social sciences and humanities publications: Description of the VIRTA-ENRESSH pilot. Helsinki: CSC & ENRESSH. Retrieved from https://doi.org/10.6084/m9.figshare.5993506PuuskaH. M.GunsR.PölönenJ.SivertsenG.Mañana-RodríguezJ.EngelsT.C.E.2018Proof of concept of a European database for social sciences and humanities publications: Description of the VIRTA-ENRESSH pilotHelsinkiCSC & ENRESSHRetrieved fromhttps://doi.org/10.6084/m9.figshare.5993506Search in Google Scholar

Publication data collection instructions for researchers (2018). Retrieved from https://wiki.eduuni.fi/display/cscsuorat/Julkaisutiedonkeruun+tutkijaohjeistukset?preview=/39984924/70200131/Publication%20data%20collection%20instructions%20for%20researchers%202018.pdfPublication data collection instructions for researchers2018Retrieved fromhttps://wiki.eduuni.fi/display/cscsuorat/Julkaisutiedonkeruun+tutkijaohjeistukset?preview=/39984924/70200131/Publication%20data%20collection%20instructions%20for%20researchers%202018.pdfSearch in Google Scholar

VIPUNEN-portal for publication statistics. Retrieved from https://vipunen.fi/en-gb/VIPUNEN-portal for publication statistics. Retrieved fromhttps://vipunen.fi/en-gb/Search in Google Scholar

JUULI-portal for browsing publication information. Retrieved from http://www.juuli.fi/JUULI-portal for browsing publication information. Retrieved fromhttp://www.juuli.fi/Search in Google Scholar

Funding implications, other uses, effects and experiencesFunding implications, other uses, effects and experiencesSearch in Google Scholar

Auranen, O., Leino, Y., Poropudas, O., & Pölönen, J. (2013). Julkaisufoorumi-luokitus ja viittausindeksit tieteellisten julkaisujen laadun mittarina. Web of Science -aineistoon perustuva vertailu (In English: Publication Forum ratings and citation scores as indicators of quality of scientific publications. A comparison based on Web of Science data), TaSTI Työraportteja 8/2013. Retrieved from http://www.uta.fi/yky/tutkimus/tasti/Julkaisut/Sahkoinenkirjasto/Julkaisufoorumi-luokitus%20ja%20viittausindeksit%20tieteellisten%20julkaisujen%20laadun%20mittareina%2019.pdfAuranenO.LeinoY.PoropudasO.PölönenJ.2013Julkaisufoorumi-luokitus ja viittausindeksit tieteellisten julkaisujen laadun mittarinaWeb of Science -aineistoon perustuva vertailu (In English: Publication Forum ratings and citation scores as indicators of quality of scientific publications. A comparison based on Web of Science data), TaSTI Työraportteja 8/2013. Retrieved fromhttp://www.uta.fi/yky/tutkimus/tasti/Julkaisut/Sahkoinenkirjasto/Julkaisufoorumi-luokitus%20ja%20viittausindeksit%20tieteellisten%20julkaisujen%20laadun%20mittareina%2019.pdfSearch in Google Scholar

Auranen, O., & Pölönen, J. (2014). Julkaisufoorumi-luokitus ja kansallinen julkaiseminen (In English: Publication Forum rating and national publishing). In Muhonen Reetta, Puuska, Hanna-Mari (eds.) Tutkimuksen kansallinen tehtävä. Tampere: Vaystapaino, 153–175.AuranenO.PölönenJ.2014Julkaisufoorumi-luokitus ja kansallinen julkaiseminen(In English: Publication Forum rating and national publishing)MuhonenReettaPuuskaHanna-MariTutkimuksen kansallinen tehtäväTampereVaystapaino153–175Search in Google Scholar

Hakala, J., & Roihuvuo, J. (2014). Research Assessment Exercise at The University of Tampere 2014: Final Report. Tampere: The University of Tampere. Retrieved from http://www.uta.fi/rae2014/index/UTA_RAE_2014_Final_Report(Web).pdfHakalaJ.RoihuvuoJ.2014Research Assessment Exercise at The University of Tampere 2014: Final ReportTampereThe University of TampereRetrieved fromhttp://www.uta.fi/rae2014/index/UTA_RAE_2014_Final_Report(Web).pdfSearch in Google Scholar

Lyytinen, A., Muhonen, R., Koskinen, O., & Eskola, O. (2016). Ulkomaiset tutkijat ja julkaisuaktiivisuus (In English: Foreign researchers and their publishing frequency), Opetusja kulttuuriministeriön julkaisuja 2016:16. Retrieved from http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75337/okm16.pdfLyytinenA.MuhonenR.KoskinenO.EskolaO.2016Ulkomaiset tutkijat ja julkaisuaktiivisuus (In English: Foreign researchers and their publishing frequency), Opetusja kulttuuriministeriön julkaisuja201616Retrieved fromhttp://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75337/okm16.pdfSearch in Google Scholar

Meriläinen, S. (2015). Julkaisufoorumin tasoluokitus: ovatko vaihtoehtoiset lähestymistavat ja poikkitieteellinen tutkimus uhattuina?: Puheenvuoro (In English: Publication Forum ratings: are alternative approaches and interdisciplinary research threatened?). Sukupuolentutkimus 28:3, 45–50. Retrieved from https://sunsseura.files.wordpress.com/2015/01/puheenvuoro_merilainen.pdfMeriläinenS.2015Julkaisufoorumin tasoluokitus: ovatko vaihtoehtoiset lähestymistavat ja poikkitieteellinen tutkimus uhattuina?: Puheenvuoro (In English: Publication Forum ratings: are alternative approaches and interdisciplinary research threatened?)Sukupuolentutkimus28345–50Retrieved fromhttps://sunsseura.files.wordpress.com/2015/01/puheenvuoro_merilainen.pdfSearch in Google Scholar

Ministry of Education and Culture (2015). Suomalaisen yliopistotutkimuksen tuottavuus ja vaikuttavuus: opetus- ja kulttuuriministeriön profiiliryhmän raportti (In English: Productivity and impact of Finnish university research. Report by the profile working group of the Ministry of Education and Culture), Opetus- ja kulttuuriministeriön julkaisuja 2015:5. Retrieved from http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75132/okm05.pdfMinistry of Education and Culture2015Suomalaisen yliopistotutkimuksen tuottavuus ja vaikuttavuus: opetus- ja kulttuuriministeriön profiiliryhmän raportti (In English: Productivity and impact of Finnish university researchReport by the profile working group of the Ministry of Education and Culture), Opetus- ja kulttuuriministeriön julkaisuja20155Retrieved fromhttp://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75132/okm05.pdfSearch in Google Scholar

Ministry of Education and Culture (2016). Yliopistolakiuudistuksen vaikutusten arviointi (In English: Impact evaluation of the Universities Act reform). Helsinki: Opetus- ja kulttuuriministeriö. Retrieved from https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75416/okm30.pdf?sequence=1Ministry of Education and Culture2016Yliopistolakiuudistuksen vaikutusten arviointi(In English: Impact evaluation of the Universities Act reform)HelsinkiOpetus- ja kulttuuriministeriöRetrieved fromhttps://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75416/okm30.pdf?sequence=1Search in Google Scholar

Muhonen, R., Eskola, O., Leino, Y., & Pölönen, J. (2016) Tutkimuksen monitieteisyys ja laatu (In English: The multidisciplinary nature and quality of research), Opetus- ja kulttuuriministeriön julkaisuja 2016:12. Retrieved from http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/64992/okm12.pdf?sequence=1&isAllowed=yMuhonenR.EskolaO.LeinoY.PölönenJ.2016Tutkimuksen monitieteisyys ja laatu (In English: The multidisciplinary nature and quality of research), Opetus- ja kulttuuriministeriön julkaisuja201612Retrieved fromhttp://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/64992/okm12.pdf?sequence=1&isAllowed=ySearch in Google Scholar

Pihlström, S. (2014). Academic publishing and interdisciplinarity: Finnish experiences. Human Affairs 24, 40–47. DOI: 10.2478/s13374-014-0204-2.PihlströmS.2014Academic publishing and interdisciplinarity: Finnish experiencesHuman Affairs24404710.2478/s13374-014-0204-2Open DOISearch in Google Scholar

Poropudas, O. (2018). Miksi suomalaisen tutkimuksen tuottavuus kasvoi, mutta laatu heikentyi? (In English: Why the productivity of Finnish Research Increased but quality decreased?). Informaatiotutkimus 37:1, 5–16. Retrieved from https://journal.fi/inf/article/view/70169/31174PoropudasO.2018Miksi suomalaisen tutkimuksen tuottavuus kasvoi, mutta laatu heikentyi? (In English: Why the productivity of Finnish Research Increased but quality decreased?)Informaatiotutkimus3715–16Retrieved fromhttps://journal.fi/inf/article/view/70169/31174Search in Google Scholar

Puuska, H. M. (2014). Scholarly Publishing Patterns in Finland. Tampere: University of Tampere. Retrieved from https://tampub.uta.fi/bitstream/handle/10024/95381/978-951-44-9480-2.pdf?sequence= 1.PuuskaH. M.2014Scholarly Publishing Patterns in FinlandTampereUniversity of TampereRetrieved fromhttps://tampub.uta.fi/bitstream/handle/10024/95381/978-951-44-9480-2.pdf?sequence= 1Search in Google Scholar

Pölönen, J., & Auranen, O. (2017). Julkaisupaine suomalaisessa tiedeyhteisössä (In English, Publishing pressure in Finnish research community). Informaatiotutkimus 36(2), 23–41. Retrieved from https://doi.org/10.23978/inf.65186PölönenJ.AuranenO.2017Julkaisupaine suomalaisessa tiedeyhteisössä (In English, Publishing pressure in Finnish research community)Informaatiotutkimus3622341Retrieved fromhttps://doi.org/10.23978/inf.65186Search in Google Scholar

Pölönen, J., Auranen, O., Engels, T., & Kulczycki, E. (2018). Taking national language publications into account: the case of the Finnish performance-based research funding system. STI 2018 Conference Proceedings. Retrieved from https://openaccess.leidenuniv.nl/bitstream/handle/1887/65223/STI2018_paper_43.pdf?sequence=1PölönenJ.AuranenO.EngelsT.KulczyckiE.2018Taking national language publications into account: the case of the Finnish performance-based research funding systemSTI 2018 Conference ProceedingsRetrieved fromhttps://openaccess.leidenuniv.nl/bitstream/handle/1887/65223/STI2018_paper_43.pdf?sequence=1Search in Google Scholar

Pölönen, J., & Sivertsen, G. (2017). Experiences with the rating of publication channels for the Nordic Model. With a response to a proposal for automated ratings from Saarela et al. (2016). Presentation at the 22nd Nordic Workshop on Bibliometrics and Research Policy. Retrieved from https://figshare.com/articles/Experiences_with_the_rating_of_publication_channels_for_the_Nordic_Model_With_a_response_to_a_proposal_for_automated_ratings_from_Saarela_et_al_2016_/5624731PölönenJ.SivertsenG.2017Experiences with the rating of publication channels for the Nordic ModelWith a response to a proposal for automated ratings from Saarela et al(2016)Presentation at the 22nd Nordic Workshop on Bibliometrics and Research PolicyRetrieved fromhttps://figshare.com/articles/Experiences_with_the_rating_of_publication_channels_for_the_Nordic_Model_With_a_response_to_a_proposal_for_automated_ratings_from_Saarela_et_al_2016_/5624731Search in Google Scholar

Pölönen, J., & Wahlfors, L. (2016). Local use of a national rating of publication channels in Finnish universities. Poster presented at the 21st Nordic Workshop on Bibliometrics and Research Policy. Retrieved from https://figshare.com/articles/Local_Use_of_a_National_Rating_of_Publication_Channels_in_Finnish_Universities_NWB_2016_poster_/4246541PölönenJ.WahlforsL.2016Local use of a national rating of publication channels in Finnish universitiesPoster presented at the 21st Nordic Workshop on Bibliometrics and Research PolicyRetrieved fromhttps://figshare.com/articles/Local_Use_of_a_National_Rating_of_Publication_Channels_in_Finnish_Universities_NWB_2016_poster_/4246541Search in Google Scholar

Saarela, M., Kärkkäinen, T., Lahtonen, T., & Rossi, T. (2016). Expert-based versus citation-based ranking of scholarly and scientific publication channels. Journal of Informetrics 10, 693–718. Retrieved from https://doi.org/10.1016/j.joi.2016.03.004SaarelaM.KärkkäinenT.LahtonenT.RossiT.2016Expert-based versus citation-based ranking of scholarly and scientific publication channelsJournal of Informetrics10693718Retrieved fromhttps://doi.org/10.1016/j.joi.2016.03.004Search in Google Scholar

Seuri, A., & Vartiainen, H. (2018). Yliopistojen rahoitus, kannustimet ja rakennekehitys (In English: University funding, incentives and structural development). Helsinki: Economic Policy Council. Retrieved from https://www.talouspolitiikanarviointineuvosto.fi/wordpress/wp-content/uploads/2018/01/Seuri_Vartiainen_2018-1.pdfSeuriA.VartiainenH.2018Yliopistojen rahoitus, kannustimet ja rakennekehitys(In English: University funding, incentives and structural development)HelsinkiEconomic Policy CouncilRetrieved fromhttps://www.talouspolitiikanarviointineuvosto.fi/wordpress/wp-content/uploads/2018/01/Seuri_Vartiainen_2018-1.pdfSearch in Google Scholar

Sivula, A., Suominen, J., & Reunanen, M. (2015). A1 alkuperäisartikkeli tieteellisessä aikakauslehdessä. Uusien julkaisukäytänteiden omaksuminen ihmistieteissä 2000-luvulla (In English: A1 original research article in scientific journal. Embracing new publishing practices in social sciences and humanities in the 21st century). Kasvatus & Aika 9(3), 149–171. Retrieved from http://www.kasvatus-ja-aika.fi/dokumentit/a10_sivulaal_2910151558.pdfSivulaA.SuominenJ.ReunanenM.2015A1 alkuperäisartikkeli tieteellisessä aikakauslehdessä. Uusien julkaisukäytänteiden omaksuminen ihmistieteissä 2000-luvulla (In English: A1 original research article in scientific journal. Embracing new publishing practices in social sciences and humanities in the 21st century). Kasvatus & Aika93149171Retrieved fromhttp://www.kasvatus-ja-aika.fi/dokumentit/a10_sivulaal_2910151558.pdfSearch in Google Scholar

Tähtinen, H., Rautava, P., & Happonen, R. P. (2014). EVO-pisteiden ja Julkaisufoorumin tasoluokituksen vertailu terveystutkimuksen arvioinnissa (In English: Comparison of EVO-points and Publication Forum level ratings for evaluation of health research). Suomen Lääkärilehti 69(10), 714–719.TähtinenH.RautavaP.HapponenR. P.2014EVO-pisteiden ja Julkaisufoorumin tasoluokituksen vertailu terveystutkimuksen arvioinnissa (In English: Comparison of EVO-points and Publication Forum level ratings for evaluation of health research)Suomen Lääkärilehti6910714719Search in Google Scholar

Wahlfors, L., & Pölönen, J. (2018). Julkaisufoorumi-luokituksen käyttö yliopistoissa (In English: The use of Publication Forum ratings in universties. Hallinnon tutkimus 37(1), 7–21.WahlforsL.PölönenJ.2018Julkaisufoorumi-luokituksen käyttö yliopistoissa (In English: The use of Publication Forum ratings in universtiesHallinnon tutkimus371721Search in Google Scholar

Wang, L., Vuolanto, P., & Muhonen, R. (2014). Bibliometrics in the research assessment exercise reports of Finnish universities and the relevant international perspectives, University of Tampere 2014. Retrieved from http://tampub.uta.fi/handle/10024/96266WangL.VuolantoP.MuhonenR.2014Bibliometrics in the research assessment exercise reports of Finnish universities and the relevant international perspectivesUniversity of Tampere2014Retrieved fromhttp://tampub.uta.fi/handle/10024/96266Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo