Social media and research: an assessment of the coverage of South African universities in ResearchGate, Web of Science and the Webometrics Ranking of World Universities

The emergence of social media, including social networking technologies, has had a profound impact on almost all human activities. Social media’s application in research is the most recent occurrence, as the technologies have gained prominence among researchers who regard social media as an avenue for not only strengthening their own networks, but also sharing their research. This article focuses on one of the social networking services for researchers, namely ResearchGate (RG), to assess the research visibility and impact of universities in South Africa. It also examines the correlation between the universities’ ResearchGate-based metrics and Web of Science (WoS) citation statistics on the one hand, and the Webometrics Ranking of World Universities’ (WRWU) ranking on the other. Results reveal that researchers in the top-ranking South African universities have quickly moved to embrace social media; there is a high correlation between RG and WoS in terms of their coverage of papers produced by universities in South Africa; there is also a high correlation between RG and WoS in terms of impact; and ranking of universities in RG, WoS and WRWU is similarly highly correlated. Further discussions, conclusions and recommendations are provided in the paper.


Introduction and background information
It is no longer debatable that social media has had a great impact in society.The term 'social media' refers to a variety of technologies and phenomena that are themselves still the subject of debate among enthusiasts (see Burke 2013;Cann 2011;Mulero 2012).Mulero (2012) uses the words 'social media' and 'social network' interchangeably while Cann (2011: 46) defines social media as the: online technologies and practices that people use to share opinions, insights, experiences, and perspectives.Social media can take many different forms, including text, images, audio, and video.These sites typically use technologies such as blogs, message boards, podcasts, wikis, and vlogs to allow users to interact.Cann (2011) offers that social media should be understood in the context of its application in facilitating the production and dissemination of information, and how it enables people to discuss and consume the information.Social media and, more so, social networking sites have become increasingly popular among all types of people, irrespective of their socio-economic and geopolitical status (International Telecommunication Union 2010).As of January 2014, internet penetration stood at 35%, social networking penetration had rapidly increased to 26%, and mobile penetration was 93% of the 7.1 billion people in the world (Dodaro 2014).Although there are divergent opinions on the scope of social media impact, all agree that social media have had a profound effect on society (see Jung 2014;Parrack 2012;O'Keeffe, Clarke-Pearson & Council on Communications and Media 2014).One of the most recent applications of social media has been in research.

Application of social media in research: a brief overview
Research is, perhaps, the latest entrant in the social media world.Traditionally, research was developed and shared through published research articles, conference proceedings, technical reports, books and book chapters.The emergence of Information and Communications Technologies (ICT) and, more specifically, the internet have opened up more opportunities through which research can be produced and shared.Several people have proposed how social media can be used in research (for example, Cann 2011;Miah 2013).Miah (2013) observes that social media can be used not only in promoting research, but in its development.In his argument, Miah maintains that it is now possible for researchers to digest a lot more content from journals which are delivered through social media than those delivered through traditional means.For instance, he argues that RSS feeds provide instant access to content of any given research article published in journals making use of social media.Time is therefore saved through the use of social media platforms such as RSS feeds.
Although the biggest users of social media are people aged between 18 and 49 (over 80% of internet users), the uptake of social media by students for research towards attaining postgraduate degrees (master's and doctoral degrees) is still minimal.According to the report by Carpenter, Wetheridge and Tanner (2012), entitled Researchers of tomorrow: the research behaviour of Generation Y doctoral students, students tend to use technology applications and social media in their research if they augment, and can easily be absorbed into, existing work practices.The report further reveals that levels of use of social media and other applications helpful in retrieving and managing research information are steadily rising among Generation Y doctoral students, but those applications most useful for collaboration and scholarly communications remain among the least used.In respect of the purpose for which researchers use social media, Cann (2011) identifies four areas:


identification of knowledge (for example, undertaking literature reviews using peer-reviewed sources);  creation of knowledge by professional researchers usually behind closed doors;  quality assurance of knowledge (for example, peer review, filtering the best for publication);  dissemination of knowledge (for example, publication, presentation at conferences).
Of late, the use of social media in research has also been extended to the analysis of the impact of research.Research evaluation has become a major player in so many aspects of being a researcher, including impacting on employment, tenure, promotion, funding and rating.The evaluation of research impact on social media has led to the introduction of alternative metrics (simply referred to as altmetrics).Altmetrics, which is sometimes called social web metrics (Costas, Zahedi & Wouters 2014) and/or influmetrics (Cronin & Weaver 1995;Rousseau & Ye 2013), is concerned with assessing how many times an output (for example, article, website, blog, dataset, grey literature, software) has been viewed, downloaded, cited, reused, shared, bookmarked and/or commented upon (Konkiel 2012).Rogers (2015) indicates that a great number of researchers have taken to social media to share their professional identities and full-text publications.In their study entitled 'Social networks, learning, and flexibility: sourcing scientific knowledge in new biotechnology firms', Liebeskind et al. (1996: 428) reveal that scientists use social media to exchange scientific knowledge and, by so doing, they "increase both their learning and their flexibility in ways that would not be possible within a self-contained hierarchical organization".Bianchini (2012) argues that researchers use social networks frequently to maintain and develop professional relationships.He believes that researchers or scientists consider social ties to be very important in their professional careers.They are driven by the fact that they wish to keep abreast of what research their colleagues are engaged in for purposes of establishing collaboration in common fields of interest and knowledge sharing.Communicating research findings is also one of the areas in which social media has found a role to play.Allen et al. (2013) assert that social media has increased dissemination of original articles in the domain of clinical pain sciences.The authors conducted a study aimed at quantifying the impact on views and downloads of articles in clinical pain sciences when they were released via a blog and on social media.They found that there was an increase in both outcome variables in the week after the blog post and social media release of the articles.They noted the following: The mean ± SD rate of HTML views in the week after the social media release was 18 ± 18 per day, whereas the rate during the other three weeks was no more than 6 ± 3 per day.The mean ± SD rate of PDF downloads in the week after the social media release was 4 ± 4 per day, whereas the rate during the other three weeks was less than 1 ± 1 per day (Allen et al., 2013: 1).
Despite the value attached to social media, especially regarding its potential to increase the dissemination of research findings, Allen et al. (2013) argue that little has been written to support their claim.

Purpose of the study
The purpose of the study was to determine and compare the visibility and impact of research produced by South African universities on ResearchGate (RG), on the one hand, and the Web of Science (WoS), on the other.Furthermore, the study sought to compare the ranking of the universities using the data extracted from RG, WoS and Webometrics Ranking of World Universities (WRWU) with a view to determining rank correlations.
In order to fulfil the above purpose, the following research questions were formulated:


To what extent have researchers in South African universities embraced social media and, more specifically, RG?  Is there a significant difference between RG and WoS in terms of their coverage of papers published by researchers in South African universities? Is there a significant difference between WoS and RG in terms of the research impact of each of the universities in South Africa? Is there a significant difference between RG and WoS on the one hand, and WRWU on the other, in terms of the ranking of South African universities?
4 Methods and materials The data was extracted from multiple sources, namely, RG; the three citation indexes (Science Citation Index, Social Sciences Citation Index and Arts and Humanities Citation Index) of WoS; and WRWU (January 2014 edition).WoS is an online scientific citation indexing service maintained by Thomson Reuters.Through its citation indexes, the service provides citation statistics of papers and data published in selected journals.All twenty-three public universities in South Africa were included in the study (see Table 1).In the case of WoS, the search was conducted using the advanced search platform.The search query, CU=South Africa, was used in WoS to extract all documents published by authors whose country affiliation was South Africa.In other words, the search was meant to yield all publications published in South Africa.The search was limited to publications between 2008 and 2013.The basis for setting the lowest (earliest) time limiter to 2008 was to align the date with the founding of RG.It should be noted that there is a possibility that the uptake of RG by academics/researchers in South African universities was not immediate, but for purposes of uniformity in the search, it was deemed necessary to limit the search within WoS to papers published between 2008 and 2013.The study also limited its search for data to only three (out of seven) databases that form the WoS.The data was analysed using the WoS in-built 'analyse' option in order to identify the participating organisations or institutions.The publications produced by each South African university were thereafter identified and isolated, per institution, for further analysis.Each university's total publications output and citation impact was obtained by subjecting the extracted data, per university, to further analysis using the built-in citation analysis function of WoS.The analysis yields the following data: total publications; total cites; cites with self-citations; cites without self-citations; average cites per paper; and h-index.Of these, total cites, average cites per paper and the h-index were used to compare research output and impact of South African universities in RG and WoS.
For the WRWU, data was extracted from the Cybermetrics Lab website (www.webometrics.info).The data was filtered by region (Africa) and then by country (South Africa).All institutions that were ranked in the report were downloaded onto a Microsoft Excel worksheet which was also used to identify the twenty-three public universities that were the subject of the current study.The relevant data downloaded for purposes of conducting the current study included: world rank of each of the South African public universities; the rank of each university in South Africa; and presence rank, impact rank, openness rank and excellence rank of each university.In order to download relevant data from RG, the statistics of South African institutions covered in RG were explored and filtered.Each university under investigation in this study was further explored to extract total publications, total RG score, and total impact points.
Data was downloaded from the three data sources in the month of February 2014.
The impact of each public university's research was examined using different indicators as provided by the different sources used to obtain data for the study.Whereas impact in RG was measured using the RG score, impact points, downloads and views, impact in WoS was examined using the sum total of citations, average citations per paper, total cites without self-citations and the h-index.The RG score and impact points were used as the two are said to measure reputation and/or impact, while the number of publications is a proxy measurement of output.It has been reported on RG's website that, although the RG score measures an individual researcher's reputation, collectively a group of colleagues' scores can be seen as a reflection of an institution.It is on the basis of this argument that the score as well as the impact points were used to compare the impact of research in RG and WoS.The raw data that was extracted from WoS and RG was reorganised and stored using Microsoft Excel.The datasets were presented in tables as reflected in Appendices A, B and C. The data was then subjected to further analysis using the Statistical Package for Social Sciences (SPSS) to obtain the correlations which were meant to aid in comparing the performance of the South African universities on social media sites (in this case, RG) and WoS on the one hand, and WRWU on the other.The correlation coefficients were obtained using Spearman's correlation.Spearman's correlation coefficient (signified by r s ) measures the strength of association between two ranked variables.The results of Spearman's correlation were plotted on tables presented as Tables 2, 4, 5 and 6.

Results
The comparison of institutions and the choice of sources of data for the study were guided by the objectives of the study, namely:


the number of papers produced by authors in South African public universities;  the reputation and/or citation impact of papers produced by South African universities;  the ranking of institutions using ResearchGate, Webometrics and the Web of Science citation indexes.

Number of publications
According to the number of papers or publications (Appendices A and C), UCT yielded the highest number in both datasets.It produced a total of 11,050 papers in RG, and a total of 6,914 publications in WoS.WITS, which was positioned second in both tables, produced 2,298 and 6,473 publications in RG and WoS respectively.In terms of the correlation between the coverage of papers in WoS and RG, Table 2 reveals that the two sources' coverage was very highly correlated, with a Spearman's correlation coefficient of r s =.886.The correlation was found to be significant, as shown in Table 2.The implication of the results in Table 2 is that there is no significant difference between the number of publications indexed in WoS and RG for South African universities, when aggregated.However, when each case is considered, the number of publications differed in the two datasets.WoS yielded a higher number of publications in most universities than RG.Out of the twenty-three universities, only eight yielded more publications in RG than WoS.These were DUT, RU, SUN, UCT, UKZN, UOVS, UP and WITS.The percentage difference between WoS and RG for each university is shown in Table 3.Of those universities which yielded the highest positive percentage difference between WoS and RG (implying a higher level of coverage in WoS as opposed to RG), UJ recorded the highest percentage (86%), followed by MUT (85%), UNIVEN (64%), NMMU (64%) and NWU (63%), to name the top five universities in Table 3.The universities that yielded more papers in RG than WoS were led by UCT, which yielded 60% more papers in RG than WoS, followed by DUT (49%), UP (44%), WITS (40%) and SUN (22%).

Reputation and/or citation impact
The citation count and/or impact performance of each university's research were compared using the following indicators: for RG, the RG score, impact points, downloads and views, and for WoS, total citations, average cites per paper, citations without self-citations and the h-index.As was the case with the number of publications, covered in both WoS and RG, the top-ranking universities in South Africa performed well in terms of the impact and/or citation count.For instance, UCT generated a total of 52,571 citations, 47,438 citations without self-citations, 7.6 average citations per paper and an hindex of 74.The rest of the top five universities performed as follows, in order of total citations, citations without selfcitations, average citations per paper and h-index: WITS (40,066; 35,737; 6.7; 65), SUN (30,287;26,441;5.19;51);UKZN (27,040;22,534;4.86;48) and UP (18,062;15,236;3.29;40).When aggregated, it was observed that the total number of citations in WoS for all universities was 220,987, while the total RG score and impact score was 3,773 and 4,760, respectively.Table 4 reveals that the average number of citations per paper as well as per university in WoS was much lower than the average number of downloads per paper and university in RG.Whereas the average number of citations per paper was 4.86, the average number of downloads per paper was 5.76.In terms of the citations and downloads per university, the figures stood at 9,608.13 and 12,530.7 respectively.The average number of views surpassed the citations and downloads per paper and university.Despite the aforementioned discrepancies between citations and RG's altmetrics (downloads and views), Table 5 reveals that there were very high correlations between the three indicators of impact.When comparing WoS and RG, it was found that the highest correlation coefficient was recorded between views and hindex (r s =.980), followed by views and cites (r s =.976), views and cites without self-citations (r s =.975), downloads and cites (r s =.974), downloads and h-index (r s =.974) and downloads and cites without self-citations (r s =.972).All these relationships were very highly correlated.

Ranking of institutions
A comparison of the ranking of universities in the three datasets (provided in Appendices A, B and C) reveals the dominance of five universities.There were, however, differences in the positions taken by the universities across the three datasets.For instance, whereas UCT occupied position one throughout in RG and WoS, it was ranked in position five in WRWU.The overall ranking of the other top universities in the order of RG, WoS and WRWU was as follows: WITS (2, 2, 8), SUN (2, 3, 1), UKZN (5, 4, 7) and UP (4, 6, 2).This pattern was replicated with the rest of the universities.
It is worth mentioning that some universities that performed relatively well in terms of WoS and RG data did poorer as far as their WRWU performance was concerned.In this category were universities such as UCT, WITS, UKZN, UJ, NWU, UWC, UOVS, TUT and NMMU.Regardless of the aforementioned individual performance of the universities, Table 6 illustrates that there were no significant differences in the ranking of the universities when comparing their coverage in the three datasets of RG, WoS and WRWU, using different performance indicators.In terms of the ranking of universities in RG and WoS, the highest correlation occurred between views and papers (r s =.986).The other highly correlated rankings in RG and WoS, in that order, occurred between downloads and papers (r s =.983), views and h-index (r s =.980), views and cites (r s =.976) and RG score and papers (r s =.976).The rankings in RG and WRWU were very highly correlated in terms of the WRWU's excellence rank and RG's views (r s =.977), downloads (r s =.967), RG score (r s =.957) and impact points (r s =.950).Similarly, the ranking of South African universities using the WoS data produced high correlations with the WRWU's ranking in terms of excellence.For instance, the correlation between the ranking of universities according to WoS's papers and h-index, on the one hand, and the WRWU's excellence rank, on the other, each produced a coefficient of r s =.984.

Discussions
The findings presented and discussed above demonstrate that the adoption and use of social media, in general and more specifically on RG, by academics/researchers in South African universities is a reality.Several academics/researchers have taken to social media to share their research findingsas reflected by the number of papers that have been posted on RG (see Appendix A).There are a substantial number of academics/researchers in each university who are occupying the RG space.This study, however, did not investigate whether the number of members per university reflects the actual number of academics in the respective universities.Nevertheless, it was noted that a substantial number have joined RG as members (see the second column in Appendix A).Perhaps the value of social media in research is becoming increasingly clearer to academics/researchers in South African universities.Miah (2013) vividly captures the importance of social media in research by stating that, with more publishers, research and peers occupying these places, opting out of social media is for academics or researchers these days akin to opting out of email in the 1990s.
A critical examination of the two extremes of Table 3 the universities at the top and bottom of the tablereveals that the majority of the universities at the bottom of the table have been identified as the most productive universities in South Africa in terms of research output and impact as well as web impact (see Onyancha 2008;Pouris & Pouris 2011;Matthews 2012;Cybermetrics Lab 2014a;Republic of South Africa. Department of Higher Education andTraining 2014 &2015).All these sources identify UCT, UP, UKZN and WITS as the most productive universities in terms of research and web impact in South Africa.The implication of the results in Table 3, as well as the fact that these universities are the most influential in South Africa in terms of research and web impact, is that researchers in these universities are more active in social media than those in the universities with low research visibility and impact.It will nevertheless be interesting to investigate the motivations that drive the researchers in the-top ranking universities in South Africa to embrace social media and, more so, RG.A critical study of the institutional policy as well as individual interests will add value to understanding the factors that have motivated researchers to join RG.
In terms of the correlation between RG and WoS in coverage of papers, the findings reveal that the two datasets correlated highly.Despite the fact that there were discrepancies in the number of papers indexed or captured for each university in the two data sources, there is a significant correlation in coverage.Does this finding vindicate WoS against accusations that the citation indexes are biased in their coverage of papers?Luwel (1999) has provided a few cases in which WoS's Science Citation Index (SCI) has been accused of bias in its coverage.Harzing (2010) notes, however, that the international coverage of papers by WoS has continued to improve but adds that "it still has a North American bias in many disciplines".Despite his critique of the international citation indexes, and more particularly WoS's citation indexes, Nwagwu (2010: 228), like Harzing (2010), notes that "autonomous databases that have regional, national and organizational focuses are beginning to emerge, but such an infrastructure is not yet available in Africa and many other developing regions".The high correlations between WoS and RG, in terms of their coverage of papers produced by researchers in the universities in South Africa, may imply that the RG uptake by South African researchers is yet to be fully embraced.In other words, for the correlation scores to be that high, a sizable number of papers produced by researchers in the universities in South Africa are not visible in both the WoS and RG.Alternatively, the high correlation scores may imply that the WoS has improved considerably in its coverage of papers produced in developing nations such as South Africa which was hitherto not adequately represented in the WoS citation indexes.
Perhaps the correlation of major indexes' data and social web metrics (altmetrics) in terms of impact is the most researched (see Moed 2005;Brody, Harnad & Carr 2006;O'Leary 2008;Thelwall & Kousha 2014;Zahedi, Costas & Wouters 2014).All the aforementioned studies, among others, have shown that altmetrics (and more particularly downloads) are in correlation with citation statistics.For instance, Zahedi, Costas and Wouters (2014) found a moderate Spearman correlation between Mendely readership counts and citation indicators in their study entitled 'How well developed are altmetrics?A cross-disciplinary analysis of the presence of "alternative metrics" in scientific publications'.There are, however, exceptions in which downloads have not been found to correlate with citation statistics (see Nieder, Dalhaug & Aandahl 2013).Nieder, Dalhaug and Aandahl (2013) investigated the correlation between article downloads and citation figures for highly accessed articles from five open access oncology journals with a hypothesis that articles with fewer downloads also accumulate fewer citations.They found a low correlation, however, and therefore concluded that downloads are not a universal surrogate for citation figures.In the current study, the average number of views surpassed the citations and downloads per paper and university.A closer examination of the number of views in each university reveals that document views were consistently higher than the number of citations and downloads.The difference in percentage ranged from 78% to 2,044% (in terms of citations) and 55% to 331% (in terms of downloads).This difference is understandable as a document can be viewed without necessarily being downloaded.It is also highly likely that a document can be downloaded but not cited; hence there were more downloads than citations.It has also been observed that "while citations take many years to accrue, tweets, Facebook shares, blog posts and reference management bookmarks tend to occur much more quickly after publication" (Mounce 2013).Allen et al. (2013: 1) concur that altmetrics are generated much faster citations.In terms of the correlation between the impact of RG and WoS, Table 5 shows that, not only was the impact in the two sources of data correlated, but it was significantly correlated.Even the average cites per paper correlated significantly with RG's impact indicators, despite recording lower correlation coefficients.The correlations associated with the average citations per paper were low due to the normalisation of the citations in respect of the number of papers covered in WoS.It should also be noted that the current study did not investigate downloads and views linked to specific papers indexed in WoS as the study did not analyse individual papers, but rather institutions.
Ranking of universities according to different performance indicators is not a new activity (Foley 2008).Foley (2008: 24) postulates that "regardless of their true ability to judge a university's success or failure, rankings are used by students, their families, and, increasingly policy makers to define the quality of institutions".Of late, global ranking systems have emerged to compare universities across geographical regions.Of these, the most commonly used include the Academic Rankings of World Universities (ARWU), World University Rankings (WUR) and WRWU (Foley 2008;Thelwall & Kousha 2014).The high correlation scores reported between the ranking of RG and WoS on the one hand, and WRWU's excellence rankings on the other, are not surprising at all, as the computation of the excellence ranking takes into consideration the excellent publications produced by institutions, "the university scientific output being part of the 10% most cited papers in their respective scientific fields" (Cybermetrics Lab, 2014b).Similar findings have been reported by Thelwall and Kousha (2014).

Conclusions
In view of the above discussion of the results, the following conclusions have been drawn:


There is no significant difference between WoS and RG in terms of the coverage of papers produced by South African universities.


There is no significant difference between RG and WoS, on the one hand, and

Correlation is significant at the 0.01 level (2-tailed). Key: CwsC -Cites without self-citations; AvC -Average cites per paper
WRWU, on the other, in terms of the ranking of South African universities across different performance indicators (number of papers, cites, cites without self-citations, average cites per paper, h-index, RG score, RG impact points, downloads, views and Webometrics rankingspresence, impact, openness and excellence).There is no significant difference between WoS and RG in terms of the research impact of each of the universities in South Africa.