An Exploratory Study of Scholarly Platforms and Features to Help Emerging Scholars Gain Visibility in the Scholars’ Community

The information present across scholarly platforms such as ResearchGate (RG), Google Scholar (GS), Academia.edu, Mendeley and Publons serves a potential base for numerous applications such as expert finding, topical authority finding, community detection, recommendation system etc. in scholarly domain.^{[1, 2]} Identifying influential scholars is a leading application in scholarly data analytic. Each scholar has a potential to outspread his/her impact across a scholarly network; however, some scholars demonstrate their dominance. These dominant scholars are known as influential.^[3] A precise assessment is required to accurately identify influential scholars.^[4] Diverse scholarly features such as count of publications, downloads, citations, recommendations, followers, followings, question, answers present on scholarly platforms provide a baseline for this assessment. Based on such features, a cumulative score can be generated and scholars with high scores are acknowledged as influential among others.

While considering diverse features for assessment, certain features demonstrate a high degree of significance. Once such features are identified, emerging scholars can focus on these features to maximize their influence and gain visibility in the scholars’ community. The scholars with high influence value can be invited for interdisciplinary explorations,^{[4, 5]} expert lectures, article reviews, scientific feedback and collaborations^{[6, 7]} based on their research domains and expertise. In this regard, influence intensification on extremely utilized scholarly platforms helps in achieving academic as well as professional impact.

This research focuses on identifying and ranking scholarly features that are significant in scholar’s assessment. Each scholarly platform has a separate feature set as well as its own user base. For our research, it is required to select a single scholarly platform that has diverse features and a wide user base. Therefore, a comparative analysis of popular scholarly platforms is conducted. The analysis reveals that RG is preferable with respect to our requirements. For RG, this research evaluates its assessment process; and hence, analytically identifies the significance of each RG feature in scholar’s assessment. Based on the identified significance, the features are assigned unique ranks.

The remainder of this paper is structured as follows: In Section 2, the related work is depicted. A comparative analysis of scholarly platforms is presented in Section 3. In Section 4, the proposed methodology is demonstrated. The results are depicted along with detailed discussion in Section 5. In Section 6, summary and future work of this research are provided.

The focus of this research is on identifying and ranking various scholarly features based on their significance in scholar’s influence assessment. The related work emphasizes on the existing influence identification methods and the features used in these methods.

The existing methods to identify influential scholars from widespread research community fall into two categories: network-based measures and statistical measures. In network-based measures, in order to find influential scholars, the collaboration networks^[8–16] of scholars are examined. The centrality algorithms such as degree, closeness, betweenness, eigenvector and PageRank are widely applied to evaluate the influence of each scholar in the network.

In statistical measures, the influence of a scholar is calculated through statistical analysis of his/her scientific contributions. h-index proposed in 2005 is pioneer in statistical measures.^[17] h-index measures the influence of a scholar through publication count and citation count. Though H-index is extremely utilized, it suffers from certain limitations: i) it is susceptible to publication time ii) it does not give any further importance to the paper once it receives h-index iii) it cannot uniquely measure the influence of a scholar. To resolve these constrains, different measures are proposed since 2005 such g-index,^[18] a-index,^[19] h²-index,^[20] m-index,^[21] r-index,^[22] ar-index,^[22] f-index,^[23] t-index,^[24] e-index,^[25] b-index,^[26] hg-index,^[27] n-index^[28] and x-index.^[29]

All these measures majorly focus on the publications and citations with different prospects. Some other measures proposed from year 2008 to 2020 include other features such as co-authorship,^[30] citing authors,^[31] citation age,^[32] research domains^{[15, 33]} and active research span^[34] along with publications and citations.

Apart from the features used in existing measures, other features such as count of recommendations, reads, projects, followers, followings, questions and answers available on scholarly platforms are equally important in assessing a scholar. It is essential to identify the significance of these features in calculating scholar’s influence. This provides scholars a vision to focus on certain aspects in order to gain increased scientific visibility in the scholars’ community. The statistical measures are at the focus of this research. The significant contributions of this paper are as follows:

To perform a comparative analysis of RG, GS, Mendeley, Academia.edu and Publons with respect to
their adoption and utilization among scholars
range and diversity of provided features
To develop a methodology for RG to
identify the significance of RG features based on their contribution in RG scholar’s influence assessment
generate ranks for significant features based on their identified significance

Five well-known scholarly platforms i.e., RG, GS, Mendeley, Academia.edu and Publons are analyzed with respect to their i) adoption and utilization ii) range and diversity of features.

Various analysis is conducted on scholarly platforms to identify their popularity in terms of total registered profiles, usage frequency and degree of activeness. Analysis was conducted on the distribution of Spanish National Research Council i.e., CSIC scholars’ profiles on Academia.edu, GS and RG.^[35] The study discovered the higher utilization of RG among all in terms of number of registered profiles. There were 4001, 2036 and 1156 CSIC scholars’ profiles registered on RG, GS and Academia. edu respectively. Survey on Academia.edu, Mendeley, RG, MyScienceWork, Humanities Common, Social Science research Network, Profology and Trellis was carried out to learn their usage frequency.^[36] The results showed substantially more frequent usage of RG among all. 26% of RG users were found to be daily users, 41% were weekly users and 18% were using RG at least monthly. In total, 85% of RG users specified using RG at least monthly. Another survey was conducted on Academia. edu, Mendeley and RG to measure the degree of activeness of scholars. The results revealed that RG has received the greatest attention in recent times.^[37] A study was carried out offering an overview of established and emerging scholarly platforms i.e., Scopus, Web of Science (WoS), PubMed, RG, GS, Academia. edu, Open Researcher and Contributor Identification (ORCID) and Publons. The results disclosed that RG is a widely utilized platform.^[1]

For mentioned platforms, no systematic analysis on the scholarly features was found in the literature. Thus, we have analyzed these platforms in detail and conducted an in-depth analysis to measure the range and diversity of features they provide. We have registered our profiles on the mentioned platforms and systematically explored their features.

Various scholarly platforms facilitate scholars to conduct diverse research-oriented activities. These activities include profile registration along with the name, affiliation, location, discipline, department, area of interest/skills/expertise, university/ organization and professional biography. Other information such as total count of reads, downloads, citations, followers, followings, co-authors, publications and credit score demonstrate an overall impact of a scholar. Scholars can display their scientific contributions in terms of publications on scholarly platforms. The publication-oriented information incorporates publication title, journals/conferences/books, publication year, co-authors, citations, reads/views, recommendations, downloads, article type, keyword list etc. Each indexed publication further has links to the article file, citing articles, references, similar or recommended articles and registered co-authors’ profiles. In many network-based scholarly platforms such as RG and Academia. edu, the links to the registered followers’ and followings’ profiles of a specific scholar are also available. Specific scholarly platform like Publons measures the impact of reviewers and editors. Links and separate count of the peer reviewed journals and editorial journals in addition to the scholar’s publication information are provided on Publons.

All such information can be extracted through various scholarly features. These features are responsible to measure the influence of a scholar on various scholarly platforms. The features offered by RG, GS, Mendeley, Academia.edu and Publons are categorized based on the type of information they provide into four categories: User Demographics, Publication Information, Link Information and Peer Review Information.

In Table 1, the features belonging to each respective category are mentioned. ✓ shows the inclusion while × displays exclusion of a specific feature on a specific platform. In our analysis, the features that are only offered by any specific platform can be considered as unique to that platform. The unique features are highlighted in Table . From thorough analysis, it is concluded that RG is preferable in terms of the mentioned two characteristics. Hence, in this research, a methodology is developed to identify and rank significant features on RG. It is noted that in this research, only user demographic features are considered for ranking as these features incisively contribute into the scholar’s influence assessment.

In this research, a methodology is proposed to identify and rank scholarly features on RG by computing feature-based influence identification. This is useful for scholars to increase their influence legitimately. The list of notations is deliberated in Table 2. The proposed methodology is displayed in Figure 1. It has five tasks: data collection, feature layer generation, feature based influence identification, similarity calculations and feature ranking. The detailed processing steps of the methodology are depicted in Algorithm 1.

In data collection task, 1544 RG scholars working in various research domains of Economics are targeted. The profiles of targeted RG scholars are collected using a web rendering method.^[38] The profile information i.e., name, affiliation, department, position, location, publication count, skills count, skills, followers count, followings count, citations count, read count, recommendation count, Q&A count, project count, RGScore and Total Research Interest (TRI) represented in terms of user demographic features are collected. The collected data is pre-processed to avoid missing value glitches and stored in a database.

In feature layer generation task, for each RG scholar, m (m=11) features contributing to influence assessment are selected from collected features. Selected m features (represented by F_i) are assigned IDs and shown in Table 3. These m features constitute the feature layer in the methodology.

In feature-based influence identification task, the significance of each RG feature present in feature layer is identified with respect to the assessment process of RG. For each RG feature, the list of RG scholars who are influential with respect to that feature is identified. The scholars having higher (feature) values are denoted as influential for that feature and are sorted in descending order of their (feature) values. For any feature, scholars obtaining higher position in the list signifies high impact towards that feature. For 11 features, 11 top k lists are generated (represented by IF_Fi) as shown in Step 1 of Algorithm 1.

In similarity calculations task, for every feature based top k (k influential RG scholars) list, the results are compared with the top k list generated from RGScore and TRI respectively. RGScore and TRI are two scores of RG to gauge the quantitative assessment of each registered and active RG scholar. RGScore is calculated based on the research in scholar’s profile and interaction of other scholars with it. TRI is mentioned as a sum of the research interest for each research item in scholars’ profiles. Similarity calculations task is demonstrated in Step 2 of Algorithm 1. The similarity values denote how similar the computed list is to the RGScore and TRI list (represented by I_RG).

In feature ranking task, based on the achieved similarity values, respective features are assigned ranks. This task is depicted in Step 3 of Algorithm 1. Higher similarity value denotes higher position of a specific feature in the rank list.

The proposed methodology is implemented on machine with Ubuntu 18.04 LTS (64-bit), 8 GB RAM and Intel Core i7-7700 processor using Python 3.8. The experimentation is performed with four values of k with identical intervals i.e., k=25, 50, 75 and 100. The results for k=25 are discussed here. Table 4 contains the list of identified top 25 influential RG scholars for each feature mentioned in Table .

The following contemplates are inferred from Table.

Each column depicts the identified top 25 influential scholars in feature-based influence list of F_i.
The identified influential scholars are ranked from 1 to 25 in the decreasing order of the values of F_i. The higher rank of a scholar R_i for F_i denotes the higher contribution of R_i towards F_i.
It is noted that a scholar R_i in dataset having the highest contribution towards a specific feature F₂ has rank 1 (position 1) in the feature-based influence list of F₂. This implies for all features.
It is less likely that scholar R_i in dataset having the highest contribution towards a specific feature F₂ will significantly contribute towards others features too.

For example, Myrna M Weissman is assigned rank 2 for features F₃ and F₁₁; rank 5 for features F₁ and F₁₀; rank 15 for feature F₂ and rank 23 for feature F₆. For other features, no significant contribution is found in top 25 experimentation.

For every feature-based influence list (IF_Fi where i=1 to 9), the similarity is calculated in comparison with two lists generated based on two features IF_F10 and IF_F11. IF_F10 and IF_F11 represent RGScore and TRI respectively. Similarity calculations are done based on the concepts of Tanimoto Coefficient, in which the ratio of the intersecting set to the union set is computed as the measure of similarity. As the aim is to calculate how close two lists (sets) are, Tanimoto Coefficient is used to perform similarity calculations.

Table 5 represents the similarity values for every pair of <IF_Fi, IF_F10> and <IF_Fi, IF_F11> for i=1 to 9. Here, IF_F10 and IF_F11 are repented as I_RG in combine. IF_Fi represents the list of identified top 25 influential RG scholars based on feature F_i. Similarity value 1 denotes identical lists whereas value 0 represents no similarity in two lists.

It is observed that top 25 influential RG scholars’ list computed based on total no. of research items is 48% similar with top 25 list received based on RGScore whereas it is 33% similar with top 25 list received based on TRI. Top 25 influential RG scholars’ list computed based on no. of Reads is 2% similar with both RGScore and TRI lists. For total no. of citations, the generated list is 28% and 9% similar to RGScore and TRI lists respectively. For total no. of followers, the generated list is 18% and 33% similar to RGScore and TRI lists respectively. For total no. of followings, the generated list is 2% similar to TRI list. For total no. of projects, the generated list is 2% similar to RGScore list. For total no. of recommendations, the generated list is 2% and 4% similar to RGScore and TRI lists respectively. For other features, there is no similarity found among the generated lists and RGScore as well as TRI.

Considering the high significance of TRI over RGScore in displaying RG scholar’s scientific contribution,^[39] the weighted similarity matrix is generated and presented in Table 6. Here, WIF_F10 and WIF_F11 represent weighted RGScore and weighted TRI while they are cumulatively labeled as WI_RG. Here, WIF_F10 = 0.5*IF_F10and WIF_F11 = IF_F11.

After calculating the weighted similarity for every <IF_Fi, WIF_F10> and <IF_Fi, WIF_F11> (for i=1 to 9) pair, the aggregated similarity i.e., ASI_FRG is computed. ASI_FRG is computed for each IF_Fi by aggregating the weighted similarity values of <IF_Fi, WIF_F10> and <IF_Fi, WIF_F11>. The aggregated similarity values lie under the range of [0, 1]. For feature ranking, the aggregated similarity values are sorted in decreasing order.

Figure 1:
The proposed methodology

Aggregated and sorted aggregated similarity values are presented in Table 7 and Table 8 respectively.

Based on the sorted aggregated similarity values for each IF_Fi, the corresponding feature F_i is assigned a rank. As per Table 9, Rank 1 denotes the highest significance and Rank 7 denotes the lowest significance of a specific feature F_i in assessing RG scholars. As the sorted aggregated similarity values for the number of questions and answers are found to be zero, they are eliminated from our ranking list.

According to the rendered RG data and obtained results for k=25, number of citations, research items, followers, reads, recommendations, followings and projects are identified as significant features in the order of their significance. Other features i.e., number of questions and answers are identified as non-significant.

The same experiment is performed with k=50, 75 and 100. Lists of identified top k influential scholars vary for different values of k; however, the obtained feature ranks are identical.

Leveraging the provided ranked features, emerging RG scholars can legitimately boost their influence and increase their visibility in the scholars’ community.

In recent times, the scholarly platforms provide a digitized medium to the scholars for performing various research-oriented activities. These scholarly platforms provide various scholarly features such as count of research items, citations, reads, recommendations, projects, questions, answers etc. By applying statistical measures on these features, an assessment score of a scholar can be computed and the scholars having higher score among others can be signified as influential.

All the scholarly features available on scholarly platforms do not imply equal significance in scholar’s assessment. To accurately measure the influence of scholars, it is essential to identify the significance of different scholarly features. This will also help scholars to focus more on certain aspects in order to boost their influence in the scholars’ community.

This research aims at identifying and ranking the significant scholarly features. For our study, it is required to select a scholarly platform with a wide range of diverse features and higher utilization among others. Thus, a comparative analysis is conducted on well-known platforms i.e., RG, GS, Mendeley, Academia.edu and Publons. The analysis revealed that RG is preferable in terms of our requirements. Thus, taking RG in consideration, a methodology is proposed to identify significant scholarly features and rank them. For the rendered RG data; number of citations, research items, followers, reads, recommendations, followings and projects (in the order of their ranking) are identified as significant features of RG.

In the future, different scholarly platforms can utilize the discovered significant features as weighted features to compute assessment scores to their users. Based on such score(s), influential scholars on scholarly platform(s) can be recognized. Common influential scholars among multiple scholarly platforms can also be recognized. Such scholars can be utilized in realistic applications of scholarly data analytic. Apart from user demographic features explored in this research, the significance of publication, link and peer review-based features can also be identified for RG and other scholarly platforms. In this research, user demographic features of RG are explored. There is a wide scope to identify the significance of publication, link and peer review-based features on RG as well as other scholarly platforms.