A Framework for the Analysis & Assessment of the Functioning of National Innovation Systems: Application on EU data

This paper proposes a framework for the analysis and assessment of the functioning of national innovation systems. Taking into account relevant literature, a framework of functions is selected for the representation of a national innovation system and certain indicators are suggested for the measurement of each function. Moreover, the proposed framework is applied for the measurement, comparison and comparative assessment of the innovation systems of all EU (European Union) countries. For this purpose, factor analysis is applied on the values of the indicators. The innovation systems of the countries are scored on the basis of the calculated factors. The results indicate that there are significant differences among European countries regarding the functioning of their innovation systems as measured by the proposed framework. Policy makers should give more emphasis on the promotion of knowledge development and diffusion, entrepreneurial R&D activity, human capital allocated to research and R&D financing.


Introduction
Although the notion of system may suggest collective and coordinated action, an innovation system is primarily a tool used to better illustrate and understand system dynamics and performance (Bergek et al., 2008). According to theory, innovation stems from the interactions of the system actors which include firms, universities, research institutes, individual researchers, market representatives, the government etc.
The most general approach to economic performance at the country level is the idea of National System of Innovation (Nelson, 1993;Lundvall, 1992;Freeman, 1995). Knowledge is produced and accumulated through an interactive and cumulative process of innovation that is embedded in the national institutional context. According to Filippetti and Archibugi (2011) the concept of national system of innovation is based on the following assumptions: a) countries exhibit systematic differences in terms of economic performance; b) economic performance depends not only on different innovation competences but also the improvement of institutions, and c) innovation policies are an effective tool for fostering the performance of countries.
There is a long stream of literature that supports the idea that innovation systems can be measured and assessed on the basis of their activities or else functions (Johnson, 1998(Johnson, , 2001Rickne, 2000;Johnson and Jacobsson, 2001;Bergek, 2002;Edquist, 2004;Liu & White, 2004;Hekkert et al., 2007). The functions of an innovation system may be defined as the processes that take place within the innovation system and contribute in technological change. The functions approach to innovation systems focuses more on the dynamics of the system rather than on the dynamics in terms of the structural components, thus enabling the separation of the system structure from the content and the formulation of policy goals in functional terms (Bergek et al., 2008). Galli and Teubal (1997) make a clear distinction between the organizations and functions of a national innovation system, since organizations increasingly have multiple roles and give emphasis on the functions and their linkages. They also distinguish between hard and soft functions. Hard functions require hard organizations (e.g. entities performing R&D), while soft functions are more related to the institutions of the system (e.g. regulatory entities) and involve catalytic and interface roles only. Examples of hard functions are: R&D activities, supply of scientific and technical services to third parties, while examples of soft functions include: diffusion of information, knowledge, and technology, design and implementation of institutions concerning patents, laws, standards etc.
According to Hekkert et al. (2007), one cannot capture the dynamics of an innovation system simply by examining the static structure of the system. Instead one should map the activities (e.g. functions) that take place within the system, since the process of change (e.g. innovation) is the combined result of many interrelated activities. However, it is impossible to map and measure all the functions of the system. Therefore, it is necessary to select the most relevant ones; those that contribute to the target of the innovation system which is to develop, apply, and diffuse new knowledge. Clearly, in the literature there is no consensus as to which functions should be included when examining an innovation system and this provides the incentive for further research. Hekkert et al. (2007) propose the following three reasons for adopting the functions approach in relation to the assessment of an innovation system. First, this approach enables the comparison of innovations systems with different institutional settings. Second, it is considered as a systematic approach with higher analytical power. Third, this method has the potential to deliver a clear set of policy targets as well as instruments to meet these targets.
The primary purpose of this paper is to propose a framework for the analysis and assessment of the functioning of national innovation systems. Taking into account previous literature, a set of functions is selected for the representation of the system. In addition, specific indicators are proposed for the measurement of each function. Then, the proposed framework of functions and indicators is applied for the measurement, comparison and assessment of the innovations systems of all EU countries. In this way an indicative subset of the initially proposed indicators can be selected; those that appear to better explain the correlations of the specific data.
For each EU country all the considered indicators are calculated on the basis of recent historical data. Then, factor analysis is applied on the values of all the indicators and a model is estimated on the basis of a subset of the initially selected indicators. Furthermore, the innovation systems of the countries are scored on the basis of the estimated factors and the results lead to further discussion.
The rest of the paper is organized as follows. Section 2, proposes the framework. Section 3 discusses the methodology and data and reports the empirical analysis results. Section 4 discusses the results. Finally, section 5 summarizes the key findings and conclusions and the policy recommendations of this study.

The Framework
In this section we describe a set of functions that could be used for the measurement, assessment and comparison of national innovation systems. Taking into account previous research, a set of functions is selected for the representation of a national innovation system and for each function a set of indicators is also considered.
In particular, the following functions are included in the framework: a. Knowledge Development, b. Entrepreneurial Experiments, c. Cooperation and Information Exchange, d. Formation of Human Capital and e. Financing and Venture Capital. Table 4 in the Appendix, presents a summary of the proposed framework including the names of the functions and the indicators suggested for every function. A short description of the content and bibliographical origins of each function is given in the following subsections.

Knowledge Development
The first function that is included in our framework is knowledge development which is considered a vital prerequisite for innovation. This function encompasses learning by searching and learning by doing (Hekkert et al., 2007). Bergek et al. (2007b) distinguish the different types of knowledge (e.g. scientific, technological, production, market, logistics and design knowledge) and the different sources of knowledge development such as R&D, learning from new applications and imitation. Galli and Teubal (1997) distinguish between hard and soft functions in relation to an innovation system. They highlight the following hard functions: a) R&D, including universities and public organizations and b) provision of scientific and technical services to third parties by industrial firms, technological centers, technical service companies, universities, governmental laboratories, and ad hoc organizations. Liu and White (2001) stress the importance of responsiveness of research conducted by public sector to the problems of manufacturers and end-users. In the case of China, the government has done so by reducing its financial support to public institutes to force them to find sources of revenue from the private sector. They have been allowed to sell the technology they develop and conduct contract research and provide consulting services to other organisations.
Changing preferences in the society can affect R&D level and priorities and change the direction of technological change (Hekkert et al., 2007). Furthermore, the volume and quality of research performed in firms and research organizations are influenced by the amount of financing available and by the availability of human capital (Rickne, 2000). Finally, network activity can be regarded as a precondition to learning by interacting. When user producer networks are concerned, it can also be regarded as learning by using (Hekkert et al., 2007).
The current level and dynamics of this function may be measured by various indicators such as the number of scientific publications, number of citations per scientific publications, number of patents etc.

Entrepreneurial Experiments
Entrepreneurs are essential for the functioning of an innovation system, as they take advantage of new business opportunities emerging from the combination of new knowledge, networks and markets. Entrepreneurs can be either new entrants in the market or existing players who diversify their business mix to take advantage of new developments. The number of active entrepreneurs is key indicator of the performance of an innovation system (Hekkert et al., 2007).
The evolution of an innovation system entails high uncertainty and volatility in terms of technologies, applications and markets (Bergek et al., 2007b). This uncertainty is a fundamental feature of technological and industrial development and is not limited to early phases in the evolution of an innovation system but also exists in later phases (Rosenberg, 1996). The only way to reduce this uncertainty is through entrepreneurial experimentation, which includes the testing of new technologies and applications.
In order for an innovation system to evolve, a whole range of firms and other organizations must choose to enter it. Therefore, there must then be sufficient incentives and no obstacles for the firms and other entities to do so (Bergek et al., 2007b). The behavior of firms and organizations is influenced by the existing institutions such as laws and regulations and standards that constitute incentives, and obstacles for innovation (Edquist, 2004).
Universities, research institutes and innovative companies represent the key actors in the national system of innovation (Acs et al., 2016). The central role of the government should be to stimulate, foster and shape the complementary of institutions and agents.
This function could be estimated and measured on the basis of the following indicative factors: business R&D expenditure, number of patent applications by the business sector, contribution of high technology sectors in the economy.

Cooperation and Information Exchange
Previous research has extensively discussed and shown that cooperation and information exchange is prerequisite for innovation (Rogers, 1983;Freeman, 1991;Pennings & Harianto, 1992;Kogut et al., 1995). Furthermore, diffusion of information has been proposed by various researchers as a very important support function for innovation systems (Johnson, 1998(Johnson, , 2001Bergek, 2002).
Private companies and especially smaller ones usually do not innovate in isolation, but in collaboration with other companies, universities, research institutes etc. (Edquist, 2004). Building successful R&D relationships is based on knowledge sharing. It is very important for firms to engage in knowledge sharing relationships with other firms, but also with organizations from other sectors of the economy (Carayannis & Alexander, 1999;Carayannis & Campbell, 2009). Furthermore, firms often support R&D at universities, the same research effort that may result in a technological opportunity to be exploited by a young company (Rickne, 2000). Given sufficient critical mass, such networks can be transformed into development blocks (e.g. synergistic clusters of firms and technologies) which create new business opportunities. (Carlsson & Stankiewicz, 1991).
For the measurement of this function the following indicators could be considered: number of international scientific co-publications, number of public-private co-publications, co-patenting involving inventors/ applicants from the same country or from various countries, collaboration of innovative firms etc.

Formation of Human Capital
Innovation is positively affected by an increase in the number of employees in research activities (Porter & Stern 2000). Education can increase the innovative power of an economy and facilitate the diffusion of knowledge which is needed to understand and process new information and to implement new technologies successfully. Competence is one of the most mentioned resources associated with innovation and with the functions of innovation (Porter, 1990;Nelson, 1992;Galli & Teubal, 1997).
The level and trends of this function can be captured and measured with the help of the following indicators: number of new doctorate graduates, % of population having completed tertiary education, R&D personnel and researchers as % of labor force etc.

Financing and Venture Capital
Financial resources are necessary as a basic input to most activities within an innovation system. Financing of innovation is primarily provided by the private sector, through stock exchanges, by venture capital organizations, or through individuals (business angels). However, the public sector also selectively provides financing or capital to innovative firms and provides funds for the research performed in universities and public research institutes.
As Rickne (2000) highlights, funding and capital are directed towards regions where there are opportunities, but only if these regions are reasonably attractive in all other dimensions as well (i.e. availability of university and company partners, high level of entrepreneurship and attractive institutions etc.).
We suggest that the level and dynamics of this function can be measured by the amount of R&D financing and venture capital investments as % of GDP.

Methodology
The empirical analysis is based on the method of exploratory factor analysis which is applied to all the indicators of each function of the framework presented in the Appendix. The values of the indicators are calculated for the sample of the 28 European countries on the basis of historical data.
The key purpose of the factor analysis is to explain the observed correlation or covariance of the variables (e.g. selected indicators) in terms of less unobserved variables called factors. The literature on factor analysis is extensive but we suggest the following useful references (Harman, 1976;Gorsuch, 1983;Tucker & MacCallum, 1997).
The analysis starts with the specification of the factor model. To this end we choose to use the correlation matrix from the series data. Moreover, the Maximum Likelihood method is used for the estimation of the model (Joreskog, 1977), while the number of factors is selected with the Velicer's (1976) minimum average partial method. Initial estimates of the common variances are required for most estimation methods. For the Maximum Likelihood method, the initial communalities are the starting values for the estimation of uniqueness. The Squared Multiple Correlation based estimates of the communalities are used.
On the basis of the aforementioned approach a series of models is estimated on the basis of the indicators of the framework. Each model comprises at least one indicator from each of the five functions of the framework. Then a set of diagnostic tests is examined to select the model that better fits the data. The selected model is the one presented in this paper.
Finally, estimates of the factors used to explain the correlation of the data are estimated on the basis of the observable data. The scores are estimated by using the exact coefficients from Thurstone's (1935) regression method and applying these coefficients to the observable data. These factor score estimates are especially useful for the discussion of the results.

Sources and data
The sample comprises data from 28 EU countries as of year 2015. Missing data points have been replaced by the latest available observation from the same source. Data have been retrieved from the databases of Eurostat, European Commission and Scopus (Eurostat databasehttp://ec.europa.eu/eurostat/data/database-last accessed July 2018, European Commission -Innovation Union Scoreboard -https://ec.europa.eu/growth/industry/innovation/factsfigures/scoreboards_en-last accessed May 2018, Scopus -https://www.scopus.com/home.uri-last accessed May 2018). Table 4 in the Appendix presents the source and the year of data per indicator. As mentioned before, the model selection process resulted in the selection of a subset the indicators that were used in the model version presented in this paper. Therefore, out of the 18 indicators of the framework, only 6 have been included in the best model that was identified. Table 1 below gives a snapshot of the selected indicators and some statistical information. For each function of the framework, one indicator has been included in the model, except from function 3 for which 2 indicators were included.

Model Estimation
Table 2 below presents the basic results of factor analysis. In the first part of the results, one can see that two factors have been retained namely F1 and F2. All the indicators appear to load on the first factor which appears to be a weighted average of the indicators. On the other hand the second factor is mainly loaded by the indicators related to the business R&D expenditure and total R&D financing and to smaller extent by the indicators related to human capital and public private cooperation in research. On the other hand, indicators related to knowledge development and international scientific co-publications have small negative impact on the second factor.
The communality and uniqueness estimates refer to the common and to the individual components which means to the part of the correlation explained by the model and to the part that remains unexplained. The two factors explain the largest part of the correlation for all indicators.
The second part of Table 2 provides information on total variance and proportion of common variance accounted for each of the factors. The first factor accounts for 83% of the common variance, while the second factor for the rest 17%.
The third part of the table presents the results of some basic tests regarding the goodness of fit of the model. Both the chi-square test (Hu and Bentler, 1995) and the Bartlett (Johnson and Wichern, 1992) corrected version of the test have p-values over 85%, indicating that the two factors satisfactory explain the correlation in the data.

Estimation of Factor Scores
The next step in the empirical analysis is to estimate the two factors from the loadings and the observable data. Table 3 presents the factor coefficients used in computing the factors and some additional indices and statistics regarding the factors and factor scores. The factor score for each country is calculated as a weighted average of the centered values of each indicator with weights the respective column of coefficients.
The indeterminacy indices that follow show that the correlation between the estimated factors and the indicators is very high. For example, the multiple correlation for the first factor is close to 100%, while the correlation for the second factor is above 98%. The minimum correlation indices are also very high.
The following parts of the table report the validity coefficients, the off-diagonal elements of the univocality matrix and the theoretical factor correlation matrix and estimated scores correlation. The validity coefficients are both in excess of Gorsuch (1983) recommended value of 0.8 and close to the maximum value. The univocality matrix reports the correlations between the factors and factor scores which should be similar to the corresponding elements of the factor correlation matrix.
Comparing the numbers we see that the univocality correlations are close to the population correlations. In addition, the estimated scores correlation matrix is close to the population factor correlation matrix. The following Figure 1 is a biplot of the estimated scores for each country together with the factor loadings which are the red lines that start from the origin and are labeled with the indicator name.
-615 - Countries with higher scores for the first factor are placed on the right part of the graph and countries with higher scores for the second factor are placed on the top of the graph. Countries that are outliers are also highlighted in the graph. For example, Germany has the highest score according to the second factor but a marginally negative score according to the first factor. On the other hand, Denmark has the highest score according to the first factor, but negative score according to the second factor. Cyprus and Luxembourg have very low scores according to the second factor.
As already mentioned, all the indicators appear to load on the first factor which also explains most of the correlation of the data. On the other hand, the second factor is mainly loaded by the business R&D expenditure and total R&D financing, while knowledge development and local diffusion seem to have a small negative impact on this factor. Figure 2 below presents the scores of the EU countries according to the two factors. The countries are ranked according to the first factor scores.

Discussion of the Results
According to the first factor the countries on the top right part of Figure 2 have the highest performance among EU countries. The scores of the first factor are decreasing as we move clockwise in the graph and become negative after Estonia. Therefore, countries such as Denmark, Sweden, Finland, and Luxembourg outperform based on the selected innovation system functions and appear to have a strong contribution to the European innovation system. On the other hand, Romania, Bulgaria, Latvia and Poland appear to have a low performance according their scores for the first factor. It should be also noted that large European economies such as Germany, France and Italy have a moderate performance in terms of the first factor which could negatively affect the innovation potential of the Union.
As it can be seen in Figure 2, the ranking of the countries is quite different based on the second factor scores. Countries with relatively moderate or even poor performance according to the first factor scores (e.g. Germany, France, Hungary, Chech Rep.) appear to outperform the others or are ranked significantly higher in terms of the second factor mainly due to the fact that in these countries business R&D activity and in general R&D financing have greater intensities as shown in the following Figure 3 which presents the values of the relevant indicators.

Sources: Eurostat. Standardised values calculated by the authors
On the other hand, there are countries with lower R&D financing than Germany and France but with greater relative to their size contribution in terms of knowledge generation and sharing as one can see in Figure 4 below. These countries usually score better in terms of the first factor.  The values of the first factor appear to be highly correlated with the innovation index values provided by the European Innovation Scoreboard which have been calculated mainly on the basis of 2015 data. Therefore, the six indicators which have been selected from the total set of indicators of the proposed framework and load on the first factor of the model appear to have similar explanatory power with the twenty-five indicators of the European Innovation Scoreboard for the year 2015. Furthermore, the second factor could be considered a supplementary axis of analysis for the innovation performance of the EU countries. We suggest that some of the countries that score better according to the second factor in comparison to the first one, are characterized by higher business R&D activity and less by their capacity to generate and diffuse knowledge. Therefore, these countries could be considered more capable of exploiting the business opportunities offered by the technological progress in EU, taking advantage of their human capital capacity and international cooperation.
-618 - Furthermore, the proposed framework is applied for the measurement, comparison and comparative assessment of the innovations systems of all EU countries. In particular, for each EU country all the considered indicators are calculated on the basis of recent historical data. Then, factor analysis is applied on the values of all the indicators and a model is estimated on the basis of a subset of the initially selected indicators. Two factors explain most of the correlation in the data. The first factor appears to be a weighted average of all six indicators included in the model and as such it could be considered as an overall innovation index. The second factor is mainly loaded by the indicators related to the business R&D expenditure and total R&D financing.
The estimated model appears to be quite representative of the data. The empirical application indicates that the proposed framework and specific indicators can be successfully used to measure, assess and compare the national innovation systems of the European countries. A comparison of the first factor scores with the innovation index scores of the countries which are provided by Innovation Union Scoreboard 2016 (European Commission, 2016) shows that the proposed framework and set of indicators could be an alternative useful and simpler approach for the comparison of the European innovation systems on the basis of the selected dataset.
Furthermore, the results of the empirical analysis indicate that there are significant differences among European countries regarding the functioning of their innovation systems as measured by the proposed framework. Policy makers should give more emphasis on the promotion of knowledge development and diffusion, entrepreneurial R&D activity, human capital allocated to research and R&D financing. It should be also noted that large European economies such as Germany, France and Italy appear to have a moderate performance in terms of the first factor which could negatively affect the innovation potential of the Union.
This work could be expanded in the following ways. First, the proposed framework could be also tested on the basis of panel data or regional data for a sample of countries. Second, the framework could be applied on the basis of time series data to examine the development of the innovation system of specific countries. Third, it could be theoretically expanded in order to take into account the interactions of the innovation system functions to the extent possible.