Recommender Systems — A Complete Guide to Machine Learning Models
Leveraging data to help users discovering new content.
Francesco Casalegno
Towards Data Science
Recommender Systems: Why And How?
Recommender systems are algorithms providing personalized suggestions for items that are most relevant to each user. With the massive growth of available online contents, users have been inundated with choices. It is therefore crucial for web platforms to offer recommendations of items to each user, in order to increase user satisfaction and engagement.
The following list shows examples of well-known web platforms with a huge number of available contents , which need efficient recommender systems to keep users interested.
- Youtube . Every minute people upload 500 hours of videos , i.e. it would take 82 years to a user to watch all videos uploaded just in the last hour.
- Spotify . Users can listen to ore than 80 million song tracks and podcasts .
- Amazon . Users can buy more than 350 million different products .
All these platforms use powerful machine learning models in order to generate relevant recommendations for each user.
Explicit Feedback vs. Implicit Feedback
In recommender systems, machine learning models are used to predict the rating rᵤᵢ of a user u on an item i . At inference time, we recommend to each user u the items l having highest predicted rating rᵤ ᵢ .
We therefore need to collect user feedback, so that we can have a ground truth for training and evaluating our models. An important distinction has to be made here between explicit feedback and implicit feedback .
Explicit feedback is a rating explicitly given by the user to express their satisfaction with an item. Examples are: number of stars on a scale from 1 to 5 given after buying a product, thumb up/down given after watching a video, etc. This feedback provides detailed information on how much a user liked an item, but it is hard to collect as most users typically don’t write reviews or give explicit ratings for each item they purchase.
Implicit feedback , on the other hand, assume that user-item interactions are an indication of preferences. Examples are: purchases/browsing history of a user, list of songs played by a user, etc. This feedback is extremely abundant , but at the same time it is less detailed and more noisy (e.g. someone may buy a product as a present for someone else). However, this noise becomes negligible when compared to the sheer size of available data of this kind, and most modern Recommender Systems tend to rely on implicit feedback .
Once we have collected explicit or implicit feedbacks, we can create the user-item rating matrix rᵤᵢ . For explicit feedback, each entry in rᵤᵢ is a numerical value—e.g. rᵤᵢ = “stars given by u to movie i ”—or “?” if user u did not rate item i . For implicit feedback, the values in rᵤᵢ are a boolean values representing presence or lack of interaction—e.g. rᵤᵢ = “did user u watch movie i ?”. Notice that the matrix rᵤᵢ is very sparse, as users interact with few items among all available contents, and they review even fewer items!
Content-Based vs. Collaborative Filtering Approaches
Recommender system can be classified according to the kind of information used to predict user preferences as Content-Based or Collaborative Filtering.
Content-Based Approach
Content-based methods describe users and items by their known metadata . Each item i is represented by a set of relevant tags—e.g. movies of the IMDb platform can be tagged as “action”, “comedy”, etc. Each user u is represented by a user profile, which can created from known user information—e.g. sex and age—or from the user’s past activity.
To train a Machine Learning model with this approach we can use a k-NN model . For instance, if we know that user u bought an item i , we can recommend to u the available items with features most similar to i .
The advantage of this approach is that items metadata are known in advance, so we can also apply it to Cold-Start scenarios where a new item or user is added to the platform and we don’t have user-item interactions to train our model. The disadvantages are that we don’t use the full set of known user-item interactions (each user is treated independently), and that we need to know metadata information for each item and user.
Collaborative Filtering Approach
Collaborative filtering methods do not use item or user metadata, but try instead to leverage the feedbacks or activity history of all users in order to predict the rating of a user on a given item by inferring interdependencies between users and items from the observed activities.
To train a Machine Learning model with this approach we typically try to cluster or factorize the rating matrix rᵤᵢ in order to make predictions on the unobserved pairs ( u,i ), i.e. where rᵤᵢ = “?”. In the following of this article we present the Matrix Factorization algorithm , which is the most popular method of this class.
The advantage of this approach is that the whole set of user-item interactions (i.e. the matrix rᵤᵢ ) is used, which typically allows to obtain higher accuracy than using Content-Based models. The disadvantage of this approach is that it requires to have a few user interactions before the model can be fitted.
Hybrid Approaches
Finally, there are also hybrid methods that try to use both the known metadata and the set of observed user-item interactions. This approach combines advantages of both Content-Based and Collaborative Filtering methods, and allow to obtain the best results. Later in this article we present LightFM , which is the most popular algorithm of this class of methods.
Collaborative Filtering: Matrix Factorization
Matrix factorization algorithms are probably the most popular and effective collaborative filtering methods for recommender systems. Matrix factorization is a latent factor model assuming that for each user u and item i there are latent vector representations pᵤ, qᵢ ∈ R ᶠ s.t. rᵤᵢ can be uniquely expressed— i.e. “factorized” — in terms of pᵤ and qᵢ . The Python library Surprise provides excellent implements of these methods.
Matrix Factorization for Explicit Feedback
The simplest idea is to model user-item interactions through a linear model . To learn the values of pᵤ and qᵢ , we can minimize a regularized MSE loss over the set K of pairs ( u , i ) for which rᵤᵢ is known. The algorithm so obtained is called probabilistic matrix factorization (PMF) .
The loss function can be minimized in two different ways. The first approach is to use stochastic gradient descent (SGD) . SGD is easy to implement, but it may have some issues because both pᵤ and qᵢ are both unknown and therefore the loss function is not convex. To solve this issue, we can alternatively fix the value pᵤ and qᵢ and obtain a convex linear regression problem that can be easily solved with ordinary least squares (OLS) . This second method is known as alternating least squares (ALS) and allows significant parallelization and speedup.
The PMF algorithm was later generalized by the singular value decomposition (SVD) algorithm, which introduced bias terms in the model. More specifically, bᵤ and bᵢ measure observed rating deviations of user u and item i , respectively, while μ is the overall average rating. These terms often explain most of the observed ratings rᵤᵢ , as some items widely receive better/worse ratings, and some users are consistently more/less generous with their ratings.
Matrix Factorization for Implicit Feedback
The SVD method can be adapted to implicit feedback datasets . The idea is to look at implicit feedback as an indirect measure of confidence . Let’s assume that the implicit feedback tᵤᵢ measures the percentage of movie i that user u has watched — e.g. tᵤᵢ = 0 means that u never watched i , tᵤᵢ = 0.1 means that he watched only 10% of it, tᵤᵢ = 2 means that he watched it twice. Intuitively, a user is more likely to be interested into a movie they watched twice, rather than in a movie they never watched. We therefore define a confidence matrix cᵤᵢ and a rating matrix rᵤᵢ as follows.
Then, we can model the observed rᵤᵢ using the same linear model used for SVD, but with a slightly different loss function. First, we compute the loss over all ( u , i ) pairs — unlike the explicit case, if user u never interacted with i we have rᵤᵢ = 0 instead of rᵤᵢ = “?” . Second, we weight each loss term by the confidence cᵤᵢ that u likes i.
Finally, the SVD++ algorithm can be used when we have access to both explicit and implicit feedbacks. This can be very useful, because typically users interact with many items (= implicit feedabck) but rate only a small subset of them (= explicit feedback). Let’s denote, for each user u , the set N(u) of items that u has interacted with. Then, we assume that an implicit interaction with an item j is associated with a new latent vector zⱼ ∈ R ᶠ . The SVD++ algorithm modifies the linear model of SVD by including into the user representation a weighted sum of these latent factors zⱼ.
Hybrid Approach: LightFM
Collaborative filtering methods based on matrix factorization often produce excellent results, but in cold-start scenarios —where little to no interaction data is available for new items and users—they cannot make good predictions because they lack data to estimate the latent factors. Hybrid approaches solve this issue by leveraging known item or user metadata in order to improve the matrix factorization model. The Python library LightFM implements one of the most popular hybrid algorithms.
In LightFM, we assume that for each user u we have collected a set of tag annotations Aᵁ(u) — e.g. “male” , “age < 30” , … — and similarly each item i has a set of annotations Aᴵ(i) — e.g. “price > 100 $” , “book” , … Then we model each user tag by a latent factor xᵁₐ ∈ R ᶠ and by a bias term bᵁₐ ∈ R , and we assume that the user vector representation pᵤ and its associated bias bᵤ can be expressed simply as the sum of these terms xᵁₐ and bᵁₐ , respectively. We take the same approach to item tags, using latent factors xᴵₐ ∈ Rᶠ and bias terms bᴵₐ ∈ R. Once we have defined pᵤ, qᵢ, bᵤ, bᵢ using these formulas, we can use the same linear model of SVD to describe the relationship between these terms and rᵤᵢ .
Notice that there are three interesting cases of this hybrid approach of LightFM.
- Cold start. If we have a new item i with known tags Aᴵ(i) , then we can use the latent vectors xᴵₐ (obtained by fitted our model on the previous data) to compute its embedding qᵢ , and therefore estimate for any user u its rating rᵤᵢ .
- No available tags. If we don’t have any known metadata for items or users, the only annotation we can use is an indicator function, i.e. a different annotation a for each user and each item. Then, user and item feature matrices are identity matrices, and LightFM reduces to a classical collaborative filtering method such as SVD.
- Content-based vs. Hybrid. If we only used user or item tags without indicator annotations, LightFM would almost be a content-based model. So in practice, to leverage user-item interactions, we also add to known tags an indicator annotation a different from each user and item.
TL;DR – Conclusions
- Recommender systems leverage machine learning algorithms to help users inundated with choices in discovering relevant contents.
- Explicit vs. implicit feedback : the first is easier to leverage, but the second is way more abundant.
- Content-based models work well in cold-start scenarios, but require to know user and item metadata .
- Collaborative filtering models typically use matrix factorization: PMF, SVD, SVD for implicit feedback, SVD++.
- Hybrid models take the best of content-based and collaborative filtering. LightFM is a great example of this approach.
- Wikipedia, Recommender System .
- “Surprise”, Python package documentation .
- (S. Funk 2006), Netflix Update: Try This at Home.
- (R. Salakhutdinov 2007), Probabilistic matrix factorization.
- ( Y. Hu 2008), Collaborative Filtering for Implicit Feedback Datasets .
- (Y. Koren 2009), Matrix Factorization Techniques for Recommender Systems .
- (Y. Koren 2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model.
- ( M. Kula 2015), Metadata Embeddings for User and Item Cold-start Recommendations .
Written by Francesco Casalegno
Machine Learning | Data Science | Statistics
Text to speech
A Survey on Modern Recommendation System based on Big Data
This survey provides an exhaustive exploration of the evolution and current state of recommendation systems, which have seen widespread integration in various web applications. It focuses on the advancement of personalized recommendation strategies for online products or services. We categorize recommendation techniques into four primary types: content-based, collaborative filtering-based, knowledge-based, and hybrid-based, each addressing unique scenarios. The survey offers a detailed examination of the historical context and the latest innovative approaches in recommendation systems, particularly those employing big data. Additionally, it identifies and discusses key challenges faced by modern recommendation systems, such as data sparsity, scalability issues, and the need for diversity in recommendations. The survey concludes by highlighting these challenges as potential areas for fruitful future research in the field.
1 Introduction
In this survey, we examine the escalating popularity and diverse application of recommendation systems in web applications, a topic extensively covered by Zhou et al. [ 1 ] . These systems, a specialized category of information filtering systems, are designed to predict user preferences for various items. They play a crucial role in guiding decision-making processes, such as purchasing decisions and music selections, as Wang et al. discuss [ 2 ] . A prime example of this application is Amazon’s personalized recommendation engine, which tailors each user’s homepage. Major companies like Amazon, YouTube, and Netflix employ these systems to enhance user experience and generate significant revenue, as noted by Adomavicius et al. and Omura et al. [ 3 , 4 ] . Figure 1 from Entezari et al. [ 5 ] illustrates a modern recommendation system. Additionally, these systems are increasingly relevant in the field of human-computer interaction (HCI), where they enhance interaction efficiency through feedback mechanisms, a topic explored in several studies [ 6 , 7 , 8 , 9 ] .
Recommendation systems are particularly crucial for certain companies, as their efficiency can lead to substantial revenue generation and competitive advantage, as evidenced in the research by Rismanto et al. and Cui et al. [ 10 , 11 ] . For instance, Netflix’s “Netflix Prize” challenge aimed to develop a recommender system surpassing their existing algorithm, with a substantial prize to incentivize innovation.
Furthermore, in the domain of big data, recommendation systems are highly prevalent, as detailed by Li et al. [ 12 , 13 ] . These systems predict user interests in purchasing based on extensive data analysis, including purchase history, ratings, and reviews. There are four widely recognized types of recommendation systems, as identified by Numnonda [ 14 ] : content-based, collaborative filtering-based, knowledge-based, and hybrid-based, each with distinct advantages and drawbacks, as Xiao et al. elucidate [ 15 ] . For example, collaborative filtering-based systems may face issues such as data sparsity and scalability, as Huang et al. mention [ 16 ] , and cold-start problems, while content-based systems might struggle to diversify user interests, as noted by Zhang et al. and Benouaret et al. [ 17 , 18 ] .
This paper is organized as follows: Section II provides a comprehensive review of both historical and modern state-of-the-art approaches in recommendation systems, coupled with an in-depth analysis of the latest advancements in the field. Section III discusses the challenges in big data-based recommendation systems, including sparsity, scalability, and diversity, and explores solutions for these challenges. The paper concludes with a summary in Section IV.
2 Recommendation Systems
Recommendation systems aim to predict users’ preferences for a certain item and provide personalized services [ 19 ] . This section will discuss several commonly used recommender methods, such as content-based method, collaborative filtering-based method, knowledge-based method, and hybrid-based method.
2.1 Content-based Recommendation Systems
The main idea of content-based recommenders is to recommend items based on the similarity between different users or items [ 20 ] . This algorithm determines and differentiates the main common attributes of a particular user’s favorite items by analyzing the descriptions of those items. Then, these preferences are stored in this user’s profile. The algorithm then recommends items with a higher degree of similarity with the user’s profile. Besides, content-based recommendation systems can capture the specific interests of the user and can recommend rare items that are of little interest to other users. However, since the feature representations of items are designed manually to a certain extent, this method requires a lot of domain knowledge. In addition, content-based recommendation systems can only recommend based on users’ existing interests, so the ability to expand users’ existing interests is limited.
2.2 Collaborative Filtering-based Recommendation Systems
Collaborative Filtering-based (CF) methods are primarily used in big data processing platforms due to their parallelization characteristics [ 21 ] . The basic principle of the recommendation system based on collaborative filtering is shown in Fig. 2 [ 22 ] . CF recommendation systems use the behavior of a group of users to recommend to other users [ 23 ] . There are mainly two types of collaborative filtering techniques, which are user-based and item-based.
User-based CF: In the user-based CF recommendation system, users will receive recommendations of products that similar users like [ 24 ] . Many similarity metrics can calculate the similarity between users or items, such as Constrained Pearson Correlation coefficient (CPC), cosine similarity, adjusted cosine similarity, etc. For example, cosine similarity is a measure of similarity between two vectors. Let x 𝑥 x italic_x and y 𝑦 y italic_y denote two vectors, cosine similarity between x 𝑥 x italic_x and y 𝑦 y italic_y can be represented by
| (1) |
| (2) |
Item-based CF: Item-based CF algorithm predicts user ratings for items based on item similarity. Generally, item-based CF yields better results than user-based CF because user-based CF suffers from sparsity and scalability issues. However, both user-based CF and item-based CF may suffer from cold-start problems [ 25 ] .
2.3 Knowledge-based Recommendation SystemsThe main idea of knowledge-based recommendation systems is to recommend items to users based on basic knowledge of users, items, and relationships between items [ 41 , 42 ] . Since knowledge-based recommendation systems do not require user ratings or purchase history, there is no cold start problem for this type of recommendation [ 43 ] . Knowledge-based recommendation systems are well suited for complex domains where items are not frequently purchased, such as cars and apartments [ 44 ] . But the acquisition of required domain knowledge can become a bottleneck for this recommendation technique [ 33 ] . 2.4 Hybrid-based Recommendation SystemsHybrid-based recommendation systems combine the advantages of multiple recommendation techniques and aim to overcome the potential weaknesses in traditional recommendation systems [ 45 ] . There are seven basic hybrid recommendation techniques [ 40 ] : weighted, mixed, switching, feature combination, feature augmentation, cascade, and meta-level methods [ 46 , 47 ] . Among all of these methods, the most commonly used is the combination of the CF recommendation methods with other recommendation methods (such as content-based or knowledge-based) to avoid sparsity, scalability, and cold-start problems [ 37 , 39 , 48 ] . 2.5 Challenges in Modern Recommendation SystemsSparsity. As we know, the usage of recommendation systems is growing rapidly. Many commercial recommendation systems use large datasets, and the user-item matrix used for filtering may be very large and sparse. Therefore, the performance of the recommendation process may be degraded due to the cold start problems caused by data sparsity [ 49 ] . Scalability. Traditional algorithms will face scalability issues as the number of users and items increases. Assuming there are millions of customers and millions of items, the algorithm’s complexity will be too large. However, recommendation systems must respond to the user’s needs immediately, regardless of the user’s rating history and purchase situation, which requires high scalability. For example, Twitter is a large web company that uses clusters of machines to scale recommendations for its millions of users [ 38 ] . Diversity. Recommendation systems also need to increase diversity to help users discover new items. Unfortunately, some traditional algorithms may accidentally do the opposite because they always recommend popular and highly-rated items that some specific users love. Therefore, new hybrid methods need to be developed to improve the performance of the recommendation systems [ 50 ] . 3 Recommendation System based on Big DataBig data refers to the massive, high growth rate and diversified information [ 51 , 52 ] . It requires new processing models to have stronger decision-making and process optimization capabilities [ 53 ] . Big data has its unique “4V” characteristics, as shown in Fig. 3 [ 54 ] : Volume, Variety, Velocity, and Veracity.  3.1 Big Data Processing FlowBig data comes from many sources, and there are many methods to process it [ 55 ] . However, the primary processing of big data can be divided into four steps [ 56 ] . Besides, Fig. 4 presents the basic flow of big data processing. Data Collection. Data Processing and Integration. The collection terminal itself already has a data repository, but it cannot accurately analyze the data. The received information needs to be pre-processed [ 57 ] . Data Analysis. In this process, these initial data are always deeply analyzed using cloud computing technology [ 58 ] . Data Interpretation.  3.2 Modern Recommendation Systems based on the Big DataThe shortcomings of traditional recommendation systems mainly focus on insufficient scalability and parallelism [ 59 ] . For small-scale recommendation tasks, a single desktop computer is sufficient for data mining goals, and many techniques are designed for this type of problems [ 60 ] .  However, the rating data is usually so large for medium-scale recommendation systems that it is impossible to load all the data into memory at once [ 61 ] . Common solutions are based on parallel computing or collective mining, sampling and aggregating data from different sources, and using parallel computing programming to perform the mining process [ 62 ] . The big data processing framework will rely on cluster computers with high-performance computing platforms [ 63 ] . At the same time, data mining tasks will be deployed on a large number of computing nodes (i.e., clusters) by running some parallel programming tools [ 64 ] , such as MapReduce [ 52 , 65 ] . For example, Fig. 5 is the MapReduce in the Recommendation Systems. In recent years, various big data platforms have emerged [ 66 ] . For example, Hadoop and Spark [ 52 ] , both developed by the Apache Software Foundation, are widely used open-source frameworks for big data architectures [ 52 , 67 ] . Each framework contains an extensive ecosystem of open-source technologies that prepare, process, manage and analyze big data sets [ 68 ] . For example, Fig. 6 is the ecosystem of Apache Hadoop [ 69 ] .  Hadoop allows users to manage big data sets by enabling a network of computers (or “nodes”) to solve vast and intricate data problems. It is a highly scalable, cost-effective solution that stores and processes structured, semi-structured and unstructured data. Spark is a data processing engine for big data sets. Like Hadoop, Spark splits up large tasks across different nodes. However, it tends to perform faster than Hadoop, and it uses random access memory (RAM) to cache and process data instead of a file system. This enables Spark to handle use cases that Hadoop cannot. The following are some benefits of the Spark framework: It is a unified engine that supports SQL queries, streaming data, machine learning (ML), and graph processing. It can be 100x faster than Hadoop for smaller workloads via in-memory processing, disk data storage, etc. It has APIs designed for ease of use when manipulating semi-structured data and transforming data.  Furthermore, Spark is fully compatible with the Hadoop eco-system and works smoothly with Hadoop Distributed File System (HDFS), Apache Hive, and others. Thus, when the data size is too big for Spark to handle in-memory, Hadoop can help overcome that hurdle via its HDFS functionality. Fig. 7 is a visual example of how Spark and Hadoop can work together. Fig. 8 is the the architecture of the modern recommendation system based on Spark.  Recommendation systems have become very popular in recent years and are used in various web applications. Modern recommendation systems aim at providing users with personalized recommendations of online products or services. Various recommendation techniques, such as content-based, collaborative filtering-based, knowledge-based, and hybrid-based recommendation systems, have been developed to fulfill the needs in different scenarios. This paper presents a comprehensive review of historical and recent state-of-the-art recommendation approaches, followed by an in-depth analysis of groundbreaking advances in modern recommendation systems based on big data. Furthermore, this paper reviews the issues faced in modern recommendation systems such as sparsity, scalability, and diversity and illustrates how these challenges can be transformed into prolific future research avenues.
Deep Learning for Recommender Systems: A Netflix Case Study
Deep learning has profoundly impacted many areas of machine learning. However, it took a while for its impact to be felt in the field of recommender systems. In this article, we outline some of the challenges encountered and lessons learned in using deep learning for recommender systems at Netflix. We first provide an overview of the various recommendation tasks on the Netflix service. We found that different model architectures excel at different tasks. Even though many deep-learning models can be understood as extensions of existing (simple) recommendation algorithms, we initially did not observe significant improvements in performance over well-tuned non-deep-learning approaches. Only when we added numerous features of heterogeneous types to the input data, deep-learning models did start to shine in our setting. We also observed that deep-learning methods can exacerbate the problem of offline–online metric (mis-)alignment. After addressing these challenges, deep learning has ultimately resulted in large improvements to our recommendations as measured by both offline and online metrics. On the practical side, integrating deep-learning toolboxes in our system has made it faster and easier to implement and experiment with both deep-learning and non-deep-learning approaches for various recommendation tasks. We conclude this article by summarizing our take-aways that may generalize to other applications beyond Netflix.  How to Cite
Information
Developed ByPart of the PKP Publishing Services Network Copyright © 2021, Association for the Advancement of Artificial Intelligence. All rights reserved.  Information
InitiativesYou are accessing a machine-readable page. In order to be human-readable, please install an RSS reader. All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess . Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications. Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers. Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal. Original Submission Date Received: .
 Article Menu
Find support for a specific problem in the support section of our website. Please let us know what you think of our products and services. Visit our dedicated information section to learn more about MDPI. JSmol ViewerSystematic review of recommendation systems for course selection.  1. Introduction2. motivation and rationale of the research, 3. research questions, 3.1. questions about the used algorithms.
3.2. Questions about the Used Dataset
3.3. Questions about the Research
4. Research Methodology4.1. title-level screening stage.
4.2. Abstract-Level Screening Stage4.3. full-text article scanning stage.
4.4. Full-Text Article Screening Stage
4.5. Data Extraction Stage5. research results, 5.1. the studies included in the slr, 5.1.1. collaborative filtering studies, 5.1.2. content-based filtering studies, 5.1.3. hybrid recommender system studies, 5.1.4. studies based on machine learning, 5.1.5. similarity-based study, 6. key studies analysis, 6.1. discussion of aims and contributions of the existing research works, 6.1.1. aim of studies that used collaborative filtering, 6.1.2. aim of studies that used content-based filtering, 6.1.3. aim of studies that used hybrid recommender systems, 6.1.4. aim of studies that used novel approaches, 6.1.5. aim of studies that used similarity-based filtering, 6.2. description of datasets used in the studies, 6.2.1. dataset description of studies that used collaborative filtering, 6.2.2. dataset description of studies that used content-based filtering, 6.2.3. dataset description of studies that used hybrid recommender systems, 6.2.4. dataset description of studies that used novel approaches.
6.2.5. Dataset Description of the Study That Used Similarity-Based Filtering6.3. research evaluation, 6.3.1. research evaluation for studies that used collaborative filtering, 6.3.2. research evaluation for studies that used content-based filtering, 6.3.3. research evaluation for studies that used hybrid recommender systems, 6.3.4. research evaluation for studies that used novel approaches, 6.3.5. research evaluation for the study that used similarity-based filtering, 7. discussion of findings, 8. gaps, challenges, future directions and conclusions for (crs) selection, 8.2. challenges, 8.3. future directions, 9. conclusions.
Author ContributionsData availability statement, conflicts of interest.
Click here to enlarge figure
Share and CiteAlgarni, S.; Sheldon, F. Systematic Review of Recommendation Systems for Course Selection. Mach. Learn. Knowl. Extr. 2023 , 5 , 560-596. https://doi.org/10.3390/make5020033 Algarni S, Sheldon F. Systematic Review of Recommendation Systems for Course Selection. Machine Learning and Knowledge Extraction . 2023; 5(2):560-596. https://doi.org/10.3390/make5020033 Algarni, Shrooq, and Frederick Sheldon. 2023. "Systematic Review of Recommendation Systems for Course Selection" Machine Learning and Knowledge Extraction 5, no. 2: 560-596. https://doi.org/10.3390/make5020033 Article MetricsArticle access statistics, further information, mdpi initiatives, follow mdpi.  Subscribe to receive issue release notifications and newsletters from MDPI journals A Novel Approach to Recommendation System Business Workflows: A Case Study for Book E-Commerce Websites
Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13381)) Included in the following conference series:
1024 Accesses 1 Citations Have you ever wondered why a song or a book or a movie becomes so popular that everyone everywhere starts talking about it? If we did not have the technology, we would say that people who love something would start recommending it to their friends and families. We live in the age of technology where there are so many algorithms that can discover the patterns of human interaction and make an excellent guess about someone’s opinion about something. These algorithms are building blocks of digital streaming services and E-Commerce websites. These services require as accurate as possible recommendation systems for them to function. While many businesses prefer one type or another of recommendation algorithms, in this study, we developed a hybrid recommendation system for a book E-Commerce website by integrating many popular classical and Deep Neural Network-based recommendation algorithms. Since explicit feedback is unavailable most of the time, all our implementations are on implicit binary feedback. The four algorithms that we were concerned about in this study were the well-known Collaborative filtering algorithms, item-based CF and user-based CF, ALS Matrix Factorization, and Deep Neural Network Based approaches. Consequently, comparing their performances and accuracy, it was not surprising that the Deep Neural Network approach was the most accurate recommender for our E-Commerce website. This is a preview of subscription content, log in via an institution to check access. Access this chapterSubscribe and save.
Tax calculation will be finalised at checkout Purchases are for personal use only Institutional subscriptions Similar content being viewed by others A Survey of Artificial Intelligence-Based E-Commerce Recommendation System E-commerce Personalized Recommendations: a Deep Neural Collaborative Filtering Approach A deep neural network-based hybrid recommender system with user-user networksGoldberg, K., et al.: EigenTaste: a constant time collaborative filtering algorithm. Inf. Retriev. 4 , 133–151 (2001) Article Google Scholar Miller, B.N., et al.: PocketLens: toward a personal recommender system. ACM Trans. Office Inf. Syst. 22 (3), 437–476 (2004) Su, X., et al.: A survey of collaborative filtering techniques. Adv. Artif. Intell. 2009 , 2 (2009) Linden, G., et al.: Amazon. com recommendations: item-to-item collaborative filtering. IEEE Internet Comput. 7 (1), 76–80 (2003) Claypool, M., et al.: Combining content-based and collaborative filters in an online newspaper. In: Proceedings of the ACM SIGIR 1999 Workshop on Recommender Systems: Algorithms and Evaluation, Berkeley, California, ACM (1999) Google Scholar Gunes, I., et al.: Shilling attacks against recommender systems: a comprehensive survey. Artif. Intell. Rev. 42 (4), 767–799 (2014) Pandey, A.K., Rajpoot, D.S.: Resolving cold start problem in recommendation system using demographic approach, pp. 213–218 (2016) Papagelis, M., et al.: Qualitative analysis of user-based and item-based prediction algorithms for recommendation agents. Eng. App. Artif. Intell. 18 (7), 781–789 (2005) Bagchi, S.: Performance and quality assessment of similarity measures in collaborative filtering using mahout. Proc. Comput. Sci. 50 , 229–234 (2015) Bokde, D., et al.: Matrix factorization model in collaborative filtering algorithms: a survey. Proc. Comput. Sci. 49 , 136–146 (2015) kumar Bokde, D., et al.: Role of matrix factorization model in collaborative filtering algorithm: a survey. ArXiv abs/1503.07475 (2015) He, X., et al.: Neural collaborative filtering. In: WWW 2017, Republic and Canton of Geneva, CHE, International World Wide Web Conferences Steering Committee, pp. 173–182 (2017) Tran, T., Cohen, R.: Hybrid recommender systems for electronic commerce. In: Proceedings of Knowledge-Based Electronic Markets, Papers from the AAAI Workshop, Technical report WS-00-04, vol. 40. AAAI Press (2000) Burke, R.: Knowledge-based recommender systems. Encycl. Libr. Inf. Syst. 69 , 175–186 (2000) Burke, R.: Hybrid recommender systems: survey and experiments. User Model. User-Adapt. Interact. 12 (4), 331–370 (2002) Smyth, B., Cotter, P.: Personalized tv listings service for the digital tv age. Knowl. Based Syst. 13 , 53–59 (2000) Basu, C., et al.: Recommendation as classification: using social and content-based information in recommendation. In: AAAI/IAAI (1998) Uzun-Per, M., et al.: Scalable recommendation systems based on finding similar items and sequences. Concurr. Comput. Pract. Exp. 2022 , e6841 (2022) Uzun-Per, M., et al.: Big data testing framework for recommendation systems in e-science and e-commerce domains. In: 2021 IEEE International Conference on Big Data (Big Data), pp. 2353–2361. IEEE (2021) Uzun-Per, M., et al.: An approach to recommendation systems using scalable association mining algorithms on big data processing platforms: a case study in airline industry. In: 2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA), pp. 1–6. IEEE (2021) Tas, K., et al.: On the implicit feedback based data modeling approaches for recommendation systems. In: 2021 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), pp. 1–6. IEEE (2021) Duzen, Z., Aktas, M.S.: An approach to hybrid personalized recommender systems. In: 2016 International Symposium on INnovations in Intelligent SysTems and Applications (INISTA), pp. 1–8. IEEE (2016) Aktas, M.S., et al.: A web based conversational case-based recommender system for ontology aided metadata discovery. In: Fifth IEEE/ACM International Workshop on Grid Computing, pp. 69–75. IEEE (2004) Arpacı, A., Aktaş, M.: Investigation of different approaches for recommendation system. In: ELECO 2018 (2018) Olmezogullari, E., Aktas, M.: Representation of click-stream datasequences for learning user navigational behavior by using embeddings. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 3173–3179. IEEE (2020) Uygun, Y., et al.: On the large-scale graph data processing for user interface testing in big data science projects. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 3173–3179. IEEE (2020) Oz, M., et al.: On the use of generative deep learning approaches for generating hidden test scripts. Int. J. Softw. Eng. Knowl. Eng. IJSEKE. 31 (10), 1447–1468 (2021) Olmezogullari, E., Aktas, M.: Pattern2vec: representation of clickstream data sequences for learning user navigational behavior. Concurr. Comput. Pract. Exp. 34 (9), e6546 (2022) Bokde, D., et al.: Role of matrix factorization model in collaborative filtering algorithm: a survey. arXiv:abs/1503.07475 (2015) Hu, Y., et al.: Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 263–272 (2008) Elkahky, A.M., et al.: A multi-view deep learning approach for cross domain user modeling in recommendation systems. In: Proceedings of the 24th International Conference on World Wide Web. WWW 2015, Republic and Canton of Geneva, CHE, International World Wide Web Conferences Steering Committee, pp. 278–288 (2015) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. CoRR abs/1412.6980 (2015) He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering source code. https://github.com/hexiangnan/neural_collaborative_filtering Bayer, I., et al.: A generic coordinate descent framework for learning from implicit feedback. In: Proceedings of the 26th International Conference on World Wide Web. WWW 2017, Republic and Canton of Geneva, CHE, International World Wide Web Conferences Steering Committee, pp. 1341–1350 (2017) He, X., et al.: Fast matrix factorization for online recommendation with implicit feedback. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR 2016, New York, NY, USA, Association for Computing Machinery, pp. 549–558 (2016) Rendle, S., et al.: BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012) He, X., et al.: Trirank: Review-aware explainable recommendation by modeling aspects. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. CIKM 2015, New York, NY, USA, Association for Computing Machinery, pp. 1661–1670 (2015) Download references AcknowledgementWe would like to thank TekhneLogos Company for helping with the dataset and the computational environment. Author informationAuthors and affiliations. Computer Engineering Department, Yildiz Technical University, Istanbul, Turkey Mounes Zaval, Said Orfan Haidari, Pinar Kosan & Mehmet S. Aktas You can also search for this author in PubMed Google Scholar Corresponding authorCorrespondence to Mounes Zaval . Editor informationEditors and affiliations. University of Perugia, Perugia, Italy Osvaldo Gervasi University of Basilicata, Potenza, Potenza, Italy Beniamino Murgante Østfold University College, Halden, Norway Sanjay Misra University of Minho, Braga, Portugal Ana Maria A. C. Rocha University of Cagliari, Cagliari, Italy Chiara Garau Rights and permissionsReprints and permissions Copyright information© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG About this paperCite this paper. Zaval, M., Haidari, S.O., Kosan, P., Aktas, M.S. (2022). A Novel Approach to Recommendation System Business Workflows: A Case Study for Book E-Commerce Websites. In: Gervasi, O., Murgante, B., Misra, S., Rocha, A.M.A.C., Garau, C. (eds) Computational Science and Its Applications – ICCSA 2022 Workshops. ICCSA 2022. Lecture Notes in Computer Science, vol 13381. Springer, Cham. https://doi.org/10.1007/978-3-031-10548-7_50 Download citationDOI : https://doi.org/10.1007/978-3-031-10548-7_50 Published : 26 July 2022 Publisher Name : Springer, Cham Print ISBN : 978-3-031-10547-0 Online ISBN : 978-3-031-10548-7 eBook Packages : Computer Science Computer Science (R0) Share this paperAnyone you share the following link with will be able to read this content: Sorry, a shareable link is not currently available for this article. Provided by the Springer Nature SharedIt content-sharing initiative
Policies and ethics
A Case Study on Various Recommendation Systems
One CitationAdaptive user interface design: a case study of web recommendation system, 12 references, amazon.com recommendations: item-to-item collaborative filtering, amazon . com recommendations item-to-item collaborative filtering, examining collaborative query reformulation: a case of travel information searching, user interactions with everyday applications as context for just-in-time information access, towards a model of collaborative information retrieval in tourism, visualizing implicit queries for information management and retrieval, implicit queries (iq) for contextualized search, just-in-time information retrieval agents, remembrance agent: a continuously running automated information retrieval system, ada and grace: direct interaction with museum visitors, related papers. Showing 1 through 3 of 0 Related Papers Book a DemoMultiply your Shopify Store's Revenue with Personalization 30 day free-trialTry Argoid for your business now! Get product recommendation ribbons like 'Trending', 'Similar Products' and more, to improve conversion and sales Try Argoid, risk-free 5 Use Case Scenarios for Recommendation Systems and How They Help SHARE THIS BLOGThe market for recommendation engines is projected to grow from USD 1.14B in 2018 to 12.03B by 2025 with a CAGR of 32.39%, for the forecasted period. These figures are an indication of the growing emphasis on customer experience while also being a byproduct of the widespread proliferation of data. On that note, here are five practical use cases of recommendation systems across different industry verticals. Use these to understand the multiple ways in which recommendation systems can add value to your business. Five Practical Use Cases of Recommendation Systems1. ecommerce recommendations.  eCommerce is by far the commonest and most frequently encountered use case of recommendation systems in action. Amazon was a pioneer in introducing this change back in 2012 by making use of item-item collaborative filtering to recommend products to the buyers. The result? A resounding 29% uplift in sales in comparison to the performance in the previous quarter! Soon enough, the recommendation engine contributed to 35% of purchases made on the platform, which was bound to impact the bottom line of the eCommerce giant. To this date, Amazon continues to remain a market leader by virtue of its helpful and user-friendly recommender engine that now also extends to the streaming platform - Amazon Prime (more on this later). The recommendation system is designed to intuitively understand and predict account interest and behaviors to drive purchases, boost engagement, increase cart volume, up-sell and cross-sell, and prevent cart abandonment. Other retailers like ASOS, Pandora, and H&M utilize recommender systems to achieve a gamut of favorable results. 2. Media Recommendations If Amazon is a frontrunner in the recommendation engine race, platforms like Netflix, Spotify, Prime Video, YouTube, and Disney+ are consolidating the role of recommendations in the field of media, entertainment, publishing, etc. Such channels have successfully normalized recommendation systems in the real world. Typically, most media streaming service providers employ a relational understanding of the type of content consumed by the user to suggest fresh content accordingly. Additionally, the self-learning and self-training aspect of AI in recommendation engines improves relevancy to maintain high levels of engagement while preventing customer churn. Consider Netflix, for example. About 75% of what users watch on Netflix is a result of its product recommendation algorithm. As a result, it is unsurprising that the platform has pegged its personalized recommendation engine at a whopping USD 1B per year as it maintains sustained subscription rates and delivers an impressive ROI that the company can redirect in fresh content creation. 3. Video Games and Stores Video games are a treasure trove of user-generated data as it contains everything - from the games they play to the choices they make. This stored repository of action, reaction, and behavior translates into usable data that allows developers to curate the experience to maximize revenue without coming off as pushy or annoying. Gaming platforms like Steam, Xbox Games Store, PlayStation Store are already well-known for their excellent recommender engines that suggest games based on the player’s gaming history, browsing history, and purchase history. As such, someone who has an interest in battle royale games like Fortnite will recommend games like PUBG, Apex Legends, and CoD rather than MMORPGs like WoW. Similarly, video games can also deploy recommender systems to nudge players towards the top of the micro-transactions funnel to make the gaming experience easier or more rewarding. Apart from boosting engagements and in-game purchases, AI-based recommender algorithms can unlock cross-selling and up-selling opportunities. 4. Location-Based Recommendations Geographic location can be a demographic factor that acts as a glue between the online and offline customer experience. It can augment marketing, advertising, and sales efforts to improve overall profitability. As a result, businesses have been in the works of developing a reliable location-based recommender system (LBRS) or a location-aware recommender system (LARS) for quite a while now, and have registered successful results. Sephora, for instance, issues geo-triggered app notifications alerting customers on existing promos and offers when they are in the vicinity of a physical store. The Starbucks app also follows a similar system for recommending happy hours and store locations. An extension of this feature is seen in Foursquare, a local search-and-discovery mobile app, which matches users with establishments like local eateries, breweries, or activity centers, based on customer location and preferences. In the process, it maintains high engagement levels and promotes businesses in the same breath. 5. Health and Fitness Health and fitness is one of the newest entrants in the recommender system but enjoys immense potential because of this lag. Applications can capture user inputs, such as their dietary preferences, activity levels, fitness goals, height and weight, BMI, etc., to suggest customized diet plans, recipes, or workout routines to match their fitness goals. In addition to logging data on such platforms, its integration with wearable devices can streamline its ability to make more accurate and valuable suggestions - such as suggesting meditation or mindfulness exercises to high-risk groups upon registering elevated vitals. Most importantly, it can capture user feedback on their fitness journey and experiences in the form of ratings to fine tune the plan and make the recommendation smarter and more personalized. Say, someone is dissatisfied with the level of difficulty of the exercise regimen, then the app can recalibrate it to suit their abilities. Final WordsRecommendations have become an implicit customer expectation as they no longer wish to sift through stores and websites or find things that they do not like. As a result, AI-based content recommendation engines, such as the one developed by Argoid , are no longer a “nice-to-have” feature but the life blood for your business. Talk to us to know more! FAQs on Recommendation System Use CasesWhat is a recommendation system. A recommendation system uses the process of information filtering to predict the products a user will like and accordingly rate the products based on users' preferences. A recommender system easily highlights the most relevant products to the users and ensures faster conversion. Why recommendation systems are essential for eCommerce stores?A recommendation system makes the shopping journey simpler and enjoyable for users. With a recommendation system, you can get product recommendations with minimum search efforts and create long-term engagement. Which recommendation system should you use?A recommendation system that you must use is - Argoid. It helps with 1:1 personalized product recommendations for your eCommerce store and ensures faster conversion and retention. Similar blogs recommended for you Revolutionizing TV Content Scheduling with AI: Introducing Argoid’s FAST Channel AI Co-PlannerLearn about Argoid's FAST channel AI Co-planner  The Rise of FAST: How Ad-Supported Streaming is Changing TVA brief on what is FAST and it is changing the television .png) Framing the Future: How Streaming Media Recommendation Engines Can Transform Your Content StrategyLearn how you can leverage streaming streaming media recommendation engines to transform your content strategy Try Argoid for your businessZero setup fee . comprehensive product . packages that suit your business..  AI-powered real-time relevance for your viewers!  Subscribe to our newsletterGet the latest on AI in eCommerce and Streaming/OTT, AI-based recommendation systems and hyper-personalization. Facebook Recommendation System Case Study Analytics Vidhya In daily life we have used many social media application like FB, IG etc. but do you think how this platform will provide the friend recommendation alert. Also, the alert for the each user is kind of unique and depends upon the his or her friend circle, office circle etc. In this case study we will take the deep look at how all this stuff will work in perspective of the Machine Learning models. Table of Contents:A. Introduction B. ML Formulation C. Business Objective and Constraints D. Data set analysis E. First Cut Approach F. Performance Matrix H. Featurization I . Modeling K. Future work M. References A. Introduction:In this case study we have to focus on the friend recommendation System for the Facebook users (i.e., recommend the person which has the highest possibility to become friend) B. ML formulation:In this case study we can use the supervised learning model with binary classification as we have to identify whether the link between the source and destination node is good or bad. C. Business Objectives and constraints:a) Objectives: • The main objective is to identify the possible missing link by using the given data. • The probability of classification is important because by using this we can select the specific number of recommendation option w.r.t the probability. b) Constraints: • Probability of prediction is useful to recommend highest probability links. • No strict latency constraint, as the objective is more about making the right decision rather than a quick decision. It would be fine and acceptable if the model takes few seconds to make a prediction. D. Data set analysis:Taken data from facebook’s recruiting challenge on Kaggle https://www.kaggle.com/c/FacebookRecruiting data contains two columns source and destination each edge in graph Data columns (total 2 columns): - Source node int64
E. First Cut Approach:1) The Dataset we have contain the nodes and edges. So, first we will do the EDA on the dataset to get clear insight of the dataset. 2) Post EDA, We have to do featurization because we will not give the datset with the node and edges directly to the model. So, to avoid this we will create the possible features by using the nodes and edges. 3) Once we have finalized the features we will create the dataset by using this features and we can use it for the modeling. 4) Now, in the modeling section we will try the different NLP model and check the performance of them. 5) Lastly we will select the model which will give the best performance. F. Performance Matrix:1) The predicted missing link should be correct so we required the high Recall value. 2) As we have to cover the maximum number of the people so the precision value should also be high. 4) So, the performance matrix F-1 score is very useful. 5) Confusion matrix Importing the important libraries:Note: We will use the networkx library as it is very helpful for graph related feature Now we will read the graph where in degree and out degree is explain as below: Let there is person A who follows the 3 person and it has the 5 followers then in this case In-degree =5 and out-degree=3. The Output of the above code is mentioned as below: The graph type is Directional Graph. Number of nodes (Source + Destination node) in the dataset is 1862220 Number of total edges in the dataset is 9437519 Both Average in degree and Average out degree for the nodes in the dataset is 5.0679 In dataset we have the nodes and edges between the nodes. The nodes are source node and destination node. So, we will analysis the data in term of the Followers, Following and Followers + Following. 1) Followers (in-degree to each person): · First, we will try to plot the graph between the number of follower vs Index No. From this we got to know that there are very less people who have the number of follower greater than 50. · To further analysis the same we will check the number of followers for the first 1.5M peoples by using the number of follower vs Index No graph. From this graph we got to know that most of the people have the no. of followers less than equal to 7. · We have tried to plot the Box-plot by using the in-degree but it does not give the clear understanding. · Now we have plotted the 90 to 100 percentile value of the in-degree and we got to know that 99 percentile value is 40 but 100 percentile value 552. · So now we will further analyze the 100-percentile value by plotting the percentile value between the 99.1 to 100. From this we got to know that 99.9 percentile value is 112 but 100 percentile value 552. · We have also plot the PDF for the in-degree data. 2) Following (out-degree from each person): · First, we will try to plot the graph between the number of following vs Index No. From this we got to know that there are very less people who have the number of following greater than 50. · To further analysis the same we will check the number of following for the first 1.5M peoples by using the number of following vs Index No graph. From this graph we got to know that most of the people have the no. of following less than equal to 7. · We have tried to plot the Box-plot by using the out-degree but it does not give the clear understanding. · Now we have plotted the 90 to 100 percentile value of the out-degree and we got to know that 99 percentile value is 40 but 100 percentile value 1566. · So now we will further analyze the 100-percentile value by plotting the percentile value between the 99.1 to 100. From this we got to know that 99.9 percentile value is 112 but 100 percentile value 1566. · The No. of person who is not following anyone or who has the no follower is 0. · No of persons those are not following anyone are 274512 and % is 14.741115442858524 · No of persons having zero followers are 188043 and % is 10.097786512871734 3) Followers + Following (in-degree and Out-degree for each person): · First, we will try to plot the graph between the number of Followers + Following vs Index No. From this we got to know that there are very less people who have the number of following greater than 80. · To further analysis the same we will check the number of Followers + Following for the first 1.5M peoples by using the number of Followers + Following vs Index No graph. From this graph we got to know that most of the people have the no. of Followers + Following less than equal to 14. · Now we have plotted the 90 to 100 percentile value of the out-degree+In-degree and we got to know that 99 percentile value is 79 but 100 percentile value 1579. · So now we will further analyze the 100-percentile value by plotting the percentile value between the 99.1 to 100. From this we got to know that 99.9 percentile value is 221 but 100 percentile value 1579. · No of persons having followers + following less than 10 are 1320326. · Min of no of followers + following is 1 and 334291 persons having minimum no of followers + following · Max of no of followers + following is 1579 and 1 persons having maximum no of followers + following · No of weakly connected components 45558 weakly connected components wit 2 nodes 32195 Currently we have the dataset with the good edges(y=1) only so the dataset is currently imbalance to make the dataset properly balance we have to add some bad edges(y=0). Generated Bad links from graph which are not in graph and whose shortest path is greater than 2. Len(U1, U4)=3 and Len(U1, U3)=2 so we will reject the (U1,U3) and accept the (U1,U4) link. The total number of the bad link(missing_edges) which we got is 9437519 which we can add in the current dataset. So, now we have the balanced dataset with the good links (y=1) and bad links (y=0) as mentioned below. Now we have to split the train and test data in the 80% and 20% combination randomly as mentioned below: Now we have done some observation about the train and test data. As shown in the dig the 81K points are present in test but absent in the train data set and 717K points are present in train but not in the test data set. So, we do not have information about this point such problem are called as the partial cold start problem. Post analyze the Output of the above code we got to know that :
H. Featurization:Currently we have the data set with the source node, destination node and Yi but we cannot give this data directly to model we need to do the featurization first on the data. Let we have the feature f1, f2, f3…etc if we input the source node and destination node to the feature f1 we will get the corresponding value for the feature and we will add this to the new feature-based data set. 1) Jaccard Distance2) cosine distance, 3) ranking measures. 3.1) Page Ranking 4) Graph Features4.1) Shortest path: 4·2) Checking for same community 4·3) Adamic/Adar Index: 4·4) Is person was following back 4·5) Katz Centrality: 4·6) Hits Score 1) Jaccard Distance:It is calculated by using the below mentioned formula: Jaccard distance we can calculate for both follower and following. It is parameter to find out the similarity between the user. Higher the Jaccard distance larger is the probability that there is the edge between the users. Let X= {u1, u2, u3} and Y= {u2, u3, u4} then |X ∩ Y|= {u1, u4} =2 and |X ∪ Y| = {u1, u2, u3, u4} =4 i.e., j=2/4=0.5 Code snippet: 2) Cosine distance:Cosine distance is also a distance same as the Jaccard distance only there is difference in the denominator. It is calculated by using the below mentioned formula: Code Snippet: 3.1) Page Ranking:If dataset have the directed graph data, then PageRank is popular feature. Consider each vertex as the web page and we can call it as the directed-graph. If any webpage has the hyperlink of CNN.com then this condition is in directed-graph condition. The Page rank score of the page is increase if it has the large number of important pages directed towards it. In below mentioned image, the B has the highest page rank score because all the web pages are directed towards it so it has the large no of web page connected and some of them are the important one. While C has the high page rank as compare to E even though it has the less page directed toward it but B is directed toward C so it is also the important web page hence it has the high page rank value as compare to the E. So, from this we got to know that the page rank is mostly depend on the quality and quantity of the web page directed toward it. It is mostly used in internet/web search task. But we can use in our data set as let consider the bill gate is person who is followed by the Mark Zugarbark, Warren Buffet (both are the celebrity like bill gates and also important person) so it will increase the score of the bill gates and also like non celebrity person like us (increases the number of follower) so we can recommend the bill gates to the any random person as there are chances that the person will follow him. We already have the page rank feature in network. We can calculate the min probability, max probability and mean probability value. The probability value is the possibility of the person land up on this page if he does any random search . Earlier we have seen that there are some nodes which are present in train but not in the test and vice versa. For such source and destination nodes we will use the mean page rank value(mean_pr). Post analyze the Output we got to know that:
4) GRAPH FEATURES:4.1) SHORTEST PATH: In this featurization we will calculate the shortest path between the two nodes. If two nodes do not have the shortest path between them then we will consider it as the -1. If two nodes are directly connected then first, we will remove this direct connection and then find out the shortest path between them. The shortest path between the source node 77697 and destination node 826021 is 10. 4.2) CHECKING FOR SAME COMMUNITY: Before understanding this feature first, we have to focus on the concept of the weekly connected component and strongly connected component. In the below mentioned image the component in section S1 is the Strongly connected component because from 1,2,0 we can travel to any component by following the direction but we cannot travel to the node 0,1,2 from node 3 and node 4. So, the component in the section S2 is weakly connected component. If we combine S1 and S2, when we try to go from vertex 4 to 0 we do not get any path to move so we cannot call the Sub graph of combination of S1 and S2 as the strongly connected component. Now let if there is some person who having set of friends in collage in one weakly connected sub-graph also called as the community and in other community is of it’s Office friend. Both the sets are very much different from each other. If there is any link between both this community then both communities can be combined We can find the weakly connected component by using the networkx library and we can check if the input nodes are weakly connected component or not. We are checking whether the source node 861 and destination node 1659750 is belong to the same community or not. But in the output we got the 0 so it shows that it does not belong to the same community. 4.3) ADAMIC/ADAR INDEX: Where N(x) →Number of inward and outward connection at node x N(y) →Number of inward and outward connection at node y We can calculate the Adar index as below: 1)If we have to find the Adar index for node x and node y then first, we have to find out the common neighborhood between them i.e., N(x) ∩ N(y) 2) There are two type of common neighborhood we can get into the node x and node y : 1) Celebrity Node: it has large number of the neighbors and node x and node y is one of them so possibility of edge between the node x and node y is very small. So, in this case log(N|u|) is large so the value of 1/ log(N|u|) is decreases. As a result, the Adar index decreases. 2)Normal Node: It has the limited neighbors. Node may be node x and node y college friend etc. so there is large possibility that there may be possible edge between the node x and node y . So, in this case log(N|u|) is small so the value of 1/ log(N|u|) is increases. As a result, Adar index increases. Calculating the Adar Index between source node 1 and destination node 189226 which is 0. 4.4) FOLLOW BACK: If there is an edge between the (a,b) then there is high probability that there is an edge between (b,a). There is possibility that source node 1 will follow back the destination node 189226 as the output is 1. 4.5) KARTZ CENTRALITY: The Katz centrality of a node is a measure of centrality in a network. It is similar as the page rank but it is very old technique so not much popular. Formula for the Katz centrality is mentioned as below: Where, Aij →Adjoin matrix with the eigen vector λ Β → controls the initial centrality and α<1/ λmax
4.6) HITS SCORE: Kindly refer the https://en.wikipedia.org/wiki/HITS_algorithm for more details.
4.7) SVD(Decomposition): Let A is the Adjacency matrix of G which is the size of the nodes in the dataset i.e., 1.78M*1.78M. It is the large sparse matrix because as shown in the below matrix A the value for the vertex i and j is Aij which is 1 if there is the edge between the ui and uj and if not then 0. So, this makes the matrix as a sparse. In below mention code, while performing the SVD we have selected the decomposition of matrix into size 6. Post applying the SVD i.e., decompose the Adjacency matrix, we got the three values U, S (also refer as the sigma) and VT so respective matrix shapes is mentioned as below. In the U the point i is define in a six possible way and also in the V same i is define in a six possible way. So, if we have two vertices then for each vertex, we have the pair of 6 feature respective to the U and VT. So, for the pair of the Vertices, we have the total 24 features with value 0 or 1. The shape of matrix post SVD decomposition is: 4.8) WEIGHT FEATURES: In order to determine the similarity of nodes, an edge weight value was calculated between nodes. Edge weight decreases as the neighbors count goes up. Intuitively, consider one million people following a celebrity on a social network then chances are most of them never met each other or the celebrity. On the other hand, if a user has 30 contacts in his/her social network, the chances are higher that many of them know each other. We have created the below mentioned feature for both train and test: · weight of incoming edges · weight of outgoing edges · weight of incoming edges + weight of outgoing edges · weight of incoming edges * weight of outgoing edges · 2*weight of incoming edges + weight of outgoing edges · weight of incoming edges + 2*weight of outgoing edges 4.9) SVD DOT: Dot product between source node SVD and destination node SVD features. as shown below. 4.10) PREFERENTIAL ATTACHMENT: One well-known concept in social networks is that users with many friends tend to create more connections in the future. This is due to the fact that in some social networks, like in finance, the rich get richer. We estimate how ” rich” our two vertices are by calculating the multiplication between the number of friends (|Γ(x)|) or followers each vertex has. It may be noted that the similarity index does not require any node neighbor information; therefore, this similarity index has the lowest computational complexity. I. Modeling:We have done with the featurization. So, now we will procced with the modeling in below mentioned steps: · Parameter Tuning · Train the model with the best parameter. · Checking the performance matrix · Feature importance Below Table will feature the performance of the different model: J. Summary:As per the performance which we have observed in the Modeling section the GBDT is the best model for the FB Recommendation system with the graph based dataset K. Future work:1. Currently we have tried to implement the important NLP algorithm but we can also check the performance of the model by using the DL method like: 1) MLP 2) LSTM 3) GRE 2. Also, we can introduce the new methods of featurization to improve the dataset as you know the thump rule of ML that more is the data better is the performance. L. Profile:Thanks for reading! do appreciate my hard work by clapping. If you want to see how I implemented this analysis , here is the project’s GitHub repo . I’m always open to constructive feedback — if you have follow-up ideas for this analysis, comment it below or reach out via LinkedIn. M. References:1) Special Thanks to: https://www.appliedaicourse.com/?gclid=CjwKCAjwzMeFBhBwEiwAzwS8zGsy_TVp9ZwwZtXRBlFlAMxT_Iq5c6CP5-dPPYQeteoT5b9jBeBmfhoCPVoQAvD_BwE 2) https://en.wikipedia.org/wiki/HITS_algorithm 3) Weight features reference: Graph-based Features for Supervised Link Prediction William Cukierski, Benjamin Hamner, Bo Yang 4)Preferential Attachment reference : https://medium.com/@cynosuremishra01/different-featurization-techniques-for-graph-related-problems-in-machine-learning-9c9d60caae60 5) https://www.kaggle.com/c/FacebookRecruiting  Written by Shubham DData Science and Machine Learning Enthusiast Text to speech  An official website of the United States government The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site. The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .
 Health Recommender Systems: Systematic ReviewRobin de croon. 1 Department of Computer Science, KU Leuven, Leuven, Belgium Leen Van HoudtNyi nyi htun, gregor Štiglic. 2 Faculty of Health Sciences, University of Maribor, Maribor, Slovenia Vero Vanden AbeeleKatrien verbert, associated data. Coded data set of all included papers. Overview of recommended items by 73 studies. Overview of evaluation approaches. Health recommender systems (HRSs) offer the potential to motivate and engage users to change their behavior by sharing better choices and actionable knowledge based on observed user behavior. We aim to review HRSs targeting nonmedical professionals (laypersons) to better understand the current state of the art and identify both the main trends and the gaps with respect to current implementations. We conducted a systematic literature review according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and synthesized the results. A total of 73 published studies that reported both an implementation and evaluation of an HRS targeted to laypersons were included and analyzed in this review. Recommended items were classified into four major categories: lifestyle, nutrition, general health care information, and specific health conditions. The majority of HRSs use hybrid recommendation algorithms. Evaluations of HRSs vary greatly; half of the studies only evaluated the algorithm with various metrics, whereas others performed full-scale randomized controlled trials or conducted in-the-wild studies to evaluate the impact of HRSs, thereby showing that the field is slowly maturing. On the basis of our review, we derived five reporting guidelines that can serve as a reference frame for future HRS studies. HRS studies should clarify who the target user is and to whom the recommendations apply, what is recommended and how the recommendations are presented to the user, where the data set can be found, what algorithms were used to calculate the recommendations, and what evaluation protocol was used. ConclusionsThere is significant opportunity for an HRS to inform and guide health actions. Through this review, we promote the discussion of ways to augment HRS research by recommending a reference frame with five design guidelines. IntroductionResearch goals. Current health challenges are often related to our modern way of living. High blood pressure, high glucose levels, and physical inactivity are all linked to a modern lifestyle characterized by sedentary living, chronic stress, or a high intake of energy-dense foods and recreational drugs [ 1 ]. Moreover, people usually make poor decisions related to their health for distinct reasons, for example, busy lifestyles, abundant options, and a lack of knowledge [ 2 ]. Practically, all modern lifestyle health risks are directly affected by people’s health decisions [ 3 ], such as an unhealthy diet or physical inactivity, which can contribute up to three-fourth of all health care costs in the United States [ 4 ]. Most risks can be minimized, prevented, or sometimes even reversed with small lifestyle changes. Eating healthily, increasing daily activities, and knowing where to find validated health information could lead to improved health status [ 5 ]. Health recommender systems (HRSs) offer the potential to motivate and engage users to change their behavior [ 6 ] and provide people with better choices and actionable knowledge based on observed behavior [ 7 - 9 ]. The overall objective of the HRS is to empower people to monitor and improve their health through technology-assisted, personalized recommendations. As one approach of modern health care is to involve patients in the cocreation of their own health, rather than just leaving it in the hands of medical experts [ 10 ], we limit the scope of this paper to HRSs that focus on laypersons, for example, nonhealth care professionals. These HRSs are different from clinical decision support systems that provide recommendations for health care professionals. However, laypersons also need to understand the rationale of recommendations, as echoed by many researchers and practitioners [ 11 ]. This paper also studies the role of a graphical user interface. To guide this study, we define our research questions (RQs) as follows: RQ1: What are the main applications of the recent HRS, and what do these HRSs recommend? RQ2: Which recommender techniques are being used across different HRSs? RQ3: How are the HRSs evaluated, and are end users involved in their evaluation? RQ4: Is a graphical user interface designed, and how is it used to communicate the recommended items to the user? Recommender Systems and TechniquesRecommender techniques are traditionally divided into different categories [ 12 , 13 ] and are discussed in several state-of-the-art surveys [ 14 ]. Collaborative filtering is the most used and mature technique that compares the actions of multiple users to generate personalized suggestions. An example of this technique can typically be found on e-commerce sites, such as “Customers who bought this item also bought...” Content-based filtering is another technique that recommends items that are similar to other items preferred by the specific user. They rely on the characteristics of the objects themselves and are likely to be highly relevant to a user’s interests. This makes content-based filtering especially valuable for application domains with large libraries of a single type of content, such as MedlinePlus’ curated consumer health information [ 15 ]. Knowledge-based filtering is another technique that incorporates knowledge by logic inferences. This type of filtering uses explicit knowledge about an item, user preferences, and other recommendation criteria. However, knowledge acquisition can also be dynamic and relies on user feedback. For example, a camera recommender system might inquire users about their preferences, fixed or changeable lenses, and budget and then suggest a relevant camera. Hybrid recommender systems combine multiple filtering techniques to increase the accuracy of recommendation systems. For example, the companies you may want to follow feature in LinkedIn uses both content and collaborative filtering information [ 16 ]: collaborative filtering information is included to determine whether a company is similar to the ones a user already followed, whereas content information ensures whether the industry or location matches the interests of the user. Finally, recommender techniques are often augmented with additional methods to incorporate contextual information in the recommendation process [ 17 ], including recommendations via contextual prefiltering, contextual postfiltering, and contextual modeling [ 18 ]. HRSs for LaypersonsRicci et al [ 12 ] define recommender systems as: Recommender Systems (RSs) are software tools and techniques providing suggestions for items to be of use to a user [ 13 , 19 , 20 ]. The suggestions relate to various decision-making processes, such as what items to buy, what music to listen to, or what online news to read. In this paper, we analyze how recommender systems have been used in health applications, with a focus on laypersons. Wiesner and Pfeifer [ 21 ] broadly define an HRS as: a specialization of an RS [recommender system] as defined by Ricci et al [ 12 ]. In the context of an HRS, a recommendable item of interest is a piece of nonconfidential, scientifically proven or at least generally accepted medical information. Researchers have sought to consolidate the vast body of literature on HRSs by publishing several surveys, literature reviews, and state-of-the-art overviews. Table 1 provides an overview of existing summative studies on HRSs that identify existing research and shows the number of studies included, the method used to analyze the studies, the scope of the paper, and their contribution. An overview of the existing health recommender system overview papers.
a HCI: human-computer interaction. b HRS: health recommender system. c CTHC: computer-tailored health communication. As can be seen in Table 1 , the scope of the existing literature varies greatly. For example, Ferretto et al [ 26 ] focused solely on HRSs in mobile apps. A total of 3 review studies focused specifically on the patient side of the HRS: (1) Calero Valdez et al [ 23 ] analyzed the existing literature from a human-computer interaction perspective and stressed the importance of a good HRS graphical user interface; (2) Schäfer et al [ 28 ] focused on tailoring recommendations to end users based on health context, history, and goals; and (3) Hors-Fraile et al [ 27 ] focused on the individual user by analyzing how HRSs can target behavior change strategies. The most extensive study was conducted by Sadasivam et al [ 29 ]. In their study, most HRSs used knowledge-based recommender techniques, which might limit individual relevance and the ability to adapt in real time. However, they also reported that the HRS has the opportunity to use a near-infinite number of variables, which enables tailoring beyond designer-written rules based on data. The most important challenges reported were the cold start [ 31 ] where limited data are available at the start of the intervention, limited sample size, adherence, and potential unintended consequences [ 29 ]. Finally, we observed that these existing summative studies were often restrictive in their final set of papers. Our contributions to the community are four-fold. First, we analyze a broader set of research studies to gain insights into the current state of the art. We do not limit the included studies to specific devices or patients in a clinical setting but focus on laypersons in general. Second, through a comprehensive analysis, we aim to identify the applications of recent HRS apps and gain insights into actionable knowledge that HRSs can provide to users (RQ1), to identify which recommender techniques have been used successfully in the domain (RQ2), how HRSs have been evaluated (RQ3), and the role of the user interface in communicating recommendations to users (RQ4). Third, based on our extensive literature review, we derive a reference frame with five reporting guidelines for future layperson HRS research. Finally, we collected and coded a unique data set of 73 papers, which is publicly available in Multimedia Appendix 1 [ 7 - 9 , 15 , 32 - 100 ] for other researchers. Search StrategyThis study was conducted according to the key steps required for systematic reviews according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [ 101 ]. A literature search was conducted using the ACM Digital Library (n=2023), IEEExplore (n=277), and PubMed (n=93) databases. As mentioned earlier, in this systematic review we focused solely on HRSs with a focus on laypersons. However, many types of systems, algorithms, and devices can be considered as a HRS. For example, push notifications in a mobile health app or health tips prompted by web services can also be considered as health-related recommendations. To outline the scope, we limited the search terms to include a recommender or recommendation, as reported by the authors. The search keywords were as follows, using an inclusive OR: ( recommender OR recommendation systems OR recommendation system ) AND (health OR healthcare OR patient OR patients ). In addition, a backward search was performed by examining the bibliographies of the survey and review papers discussed in the Introduction section and the reference list of included studies to identify any additional studies. A forward search was performed to search for articles that cited the work summarized in Table 1 . Study Inclusion and Exclusion CriteriaAs existing work did not include many studies ( Table 1 ) and focused on a specific medical domain or device, such as mobile phones, this literature review used nonrestrictive inclusion criteria. Studies that met all the following criteria were included in the review: described an HRS whose primary focus was to improve health (eg, food recommenders solely based on user preferences [ 102 ] were not included); targeted laypersons (eg, activity recommendations targeted on a proxy user such as a coach [ 103 ] were not included); implemented the HRS (eg, papers describing an HRS concept are not included); reported an evaluation, either web-based or offline evaluation; peer-reviewed and published papers; published in English. Papers were excluded when one of the following was true: the recommendations of HRSs were unclear; the full text was unavailable; or a newer version was already included. Finally, when multiple papers described the same HRS, only the latest, relevant full paper was included. ClassificationTo address our RQs, all included studies were coded for five distinct coding categories. Study DetailsTo contextualize new insights, the publication year and publication venue were analyzed. Recommended ItemsHRSs are used across different health domains. To provide details on what is recommended, all papers were coded according to their respective health domains. To not limit the scope of potential items, no predefined coding table was used. Instead, all papers were initially coded by the first author. These resulting recommendations were then clustered together in collaboration with the coauthors into four categories, as shown in Multimedia Appendix 2 . Recommender TechniquesThis category encodes the recommender techniques that were used: collaborative filtering [ 104 ], content-based filtering [ 105 ], knowledge-based filtering [ 106 ], and their hybridizations [ 107 ]. Some studies did not specify any algorithmic details or compared multiple techniques. Finally, when an HRS used contextual information, it was coded whether they used pre- or postfiltering or contextual modeling. Evaluation ApproachThis category encodes which evaluation protocols were used to measure the effect of HRSs. We coded whether the HRSs were evaluated through offline evaluations (no users involved), surveys, heuristic feedback from expert users, controlled user studies, deployments in the wild , and randomized controlled trials (RCTs). We also coded sample size and study duration and whether ethical approval was gathered and needed. Interface and TransparencyRecommender systems are often perceived as a black box , as the rationale for recommendations is often not explained to end users. Recent research increasingly focuses on providing transparency to the inner logic of the system [ 11 ]. We encoded whether explanations are provided and, in this case, how such transparency is supported in the user interface. Furthermore, we also classified whether the user interface was designed for a specific platform, categorized as mobile , web , or other. Data Extraction, Intercoder Reliability, and Quality AssessmentThe required information for all included technologies and studies was coded by the first author using a data extraction form. Owing to the large variety of study designs, the included studies were assessed for quality (detailed scores given in Multimedia Appendix 1 ) using the tool by Hawker et al [ 108 ]. Using this tool, the abstract and title , introduction and aims , method and data , sample size (if applicable), data analysis , ethics and bias , results , transferability or generalizability , and implications and usefulness were allocated a score between 1 and 4, with higher scoring studies indicating higher quality. A random selection with 14% (10/73) of the papers was listed in a spreadsheet and coded by a second researcher following the defined coding categories and subcategories. The decisions made by the second researcher were compared with the first. With the recommended items ( Multimedia Appendix 2 ), there was only one small disagreement between physical activity and leisure activity [ 32 ], but all other recommended items were rated exactly the same; the recommender techniques had a Cohen κ value of 0.71 ( P <.001) and the evaluation approach scored a Cohen κ value of 0.81 ( P <.001). There was moderate agreement (Cohen κ=0.568; P <.001) between the researchers concerning the quality of the papers. The interfaces used were in perfect agreement. Finally, the coding data are available in Multimedia Appendix 1 . The literature in three databases yielded 2340 studies, of which only 23 were duplicates and 53 were full proceedings, leaving 2324 studies to be screened for eligibility. A total of 2161 studies were excluded upon title or abstract screening because they were unrelated to health or targeted at medical professionals or because the papers did not report an evaluation. Thus, the remaining 163 full-text studies were assessed for eligibility. After the removal of 90 studies that failed the inclusion criteria or met the exclusion criteria, 73 published studies remained. The search process is illustrated in Figure 1 .  Flow diagram according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. EC: exclusion criteria; IC: inclusion criteria. All included papers were published in 2009 or later, following an upward trend of increased popularity. The publication venues of HRSs are diverse. Only the PervasiveHealth [ 33 - 35 ], RecSys [ 36 , 37 , 109 ], and WI-IAT [ 38 - 40 ] conferences published 3 papers each that were included in this study. The Journal of Medical Internet Research was the only journal that occurred more frequently in our data set; 5 papers were published by Journal of Medical Internet Research [ 41 - 45 ]. The papers were first rated using Hawker tool [ 108 ]. Owing to a large number of offline evaluations, we did not include the sample score to enable a comparison between all included studies. The papers received an average score of 24.32 (SD 4.55, max 32; data set presented in Multimedia Appendix 1 ). Most studies scored very poor on reporting ethics and potential biases, as illustrated in Figure 2 . However, there is an upward trend over the years in more adequate reporting of ethical issues and potential biases. The authors also limited themselves to their specific case studies and did not make any recommendations for policy (last box plot is presented in Figure 2 ). All 73 studies reported the use of different data sets. Although all recommended items were health related, only Asthana et al [ 46 ] explicitly mentioned using electronic health record data. Only 14% (10/73) [ 7 , 47 - 55 ] explicitly reported that they addressed the cold-start problem.  Distribution of the quality assessment using Hawker tool. Most HRSs operated in different domains and thus recommended different items. In this study, four nonmutually exclusive categories of recommended items were identified: lifestyle 33% (24/73), nutrition 36% (26/73), general health information 32% (23/73), and specific health condition–related recommendations 12% (9/73). The only significant trend we found is the increasing popularity of nutrition advice . Multimedia Appendix 2 shows the distribution of these recommended items. Many HRSs, 33% (24/73) of the included studies, suggest lifestyle-related items, but they differ greatly in their exact recommendations. Physical activity is often recommended. Physical activities are often personalized according to personal interests [ 56 ] or the context of the user [ 35 ]. In addition to physical activities, Kumar et al [ 32 ] recommend eating, shopping, and socializing activities. One study analyzes the data and measurements to be tracked for an individual and then recommends the appropriate wearable technologies to stimulate proactive health [ 46 ]. A total of 7 studies [ 7 , 9 , 42 , 53 , 57 - 59 ] more directly try to convince users to alter their behavior by recommending them to change, or alter their behavior: for example, Rabbi et al [ 7 ] learn “a user’s physical activity and dietary behavior and strategically suggests changes to those behaviors for a healthier lifestyle . ” In another example, both Marlin et al [ 59 ] and Sadasivam et al [ 42 ] motivate users to stop smoking by providing them with tailored messages, such as “Keep in mind that cravings are temporary and will pass.” Messages could reflect the theoretical determinants of quitting, such as positive outcome expectations and self-efficacy enhancing small goals [ 42 ]. The influence of food on health is also clear from the large subset of HRSs dealing with nutrition recommendations. A mere 36% (26/73) of the studies recommend nutrition-related information, such as recipes [ 50 ], meal plans [ 36 ], restaurants [ 60 ], or even help with choosing healthy items from a restaurant menu [ 61 ]. Wayman and Madhvanath [ 37 ] provide automated, personalized, and goal-driven dietary guidance to users based on grocery receipt data. Trattner and Elsweiler [ 62 ] use postfiltering to focus on healthy recipes only and extended them with nutrition advice, whereas Ge et al [ 48 ] require users to first enter their preferences for better recommendations. Moreover, Gutiérrez et al [ 63 ] propose healthier alternatives through augmented reality when the users are shopping. A total of 7 studies specifically recommend healthy recipes [ 47 , 48 , 50 , 62 , 64 - 66 ]. Most HRSs consider the health condition of the user, such as the DIETOS system [ 67 ]. Other systems recommend recipes that are synthesized based on existing recipes and recommend new recipes [ 64 ], assist parents in making appropriate food for their toddlers [ 47 ], or help users to choose allergy-safe recipes [ 65 ]. General Health InformationAccording to 32% (23/73) of the included studies, providing access to trustworthy health care information is another common objective. A total of 5 studies focused on personalized, trustworthy information per se [ 15 , 55 , 68 - 70 ], whereas 5 others focused on guiding users through health care forums [ 52 , 71 - 74 ]. In total, 3 studies [ 55 , 68 , 69 ] provided personalized access to general health information. For example, Sanchez Bocanegra et al [ 15 ] targeted health-related videos and augmented them with trustworthy information from the United States National Library of Medicine (MedlinePlus) [ 110 ]. A total of 3 studies [ 52 , 72 , 74 ] related to health care forums focused on finding relevant threads. Cho et al [ 72 ] built “an autonomous agent that automatically responds to an unresolved user query by posting an automated response containing links to threads discussing similar medical problems.” However, 2 studies [ 71 , 73 ] helped patients to find similar patients. Jiang and Yang [ 71 ] investigated approaches for measuring user similarity in web-based health social websites, and Lima-Medina et al [ 73 ] built a virtual environment that facilitates contact among patients with cardiovascular problems. Both studies aim to help users seek informational and emotional support in a more efficient way. A total of 4 studies [ 41 , 75 - 77 ] helped patients to find appropriate doctors for a specific health problem, and 4 other studies [ 51 , 78 - 80 ] focused on finding nearby hospitals. A total of 2 studies [ 78 , 79 ] simply focused on the clinical preferences of the patients, whereas Krishnan et al [ 111 ] “provide health care recommendations that include Blood Donor recommendations and Hospital Specialization.” Finally, Tabrizi et al [ 80 ] considered patient satisfaction as the primary feature of recommending hospitals to the user. Specific Health ConditionsThe last group of studies (9/73, 12%) focused on specific health conditions. However, the recommended items vary significantly. Torrent-Fontbona and Lopez Ibanez [ 81 ] have built a knowledge-based recommender system to assist diabetes patients in numerous cases, such as the estimated carbohydrate intake and past and future physical activity. Pustozerov et al [ 43 ] try to “reduce the carbohydrate content of the desired meal by reducing the amount of carbohydrate-rich products or by suggesting variants of products for replacement.” Li and Kong [ 82 ] provided diabetes-related information, such as the need for a low-sodium lunch, targeted on American Indians through a mobile app. Other health conditions supported by recommender systems include depression and anxiety [ 83 ], mental disorders [ 45 ], and stress [ 34 , 54 , 84 , 85 ]. Both the mental disorder [ 45 ] and the depression and anxiety [ 83 ] HRSs recommend mobile apps. For example, the app MoveMe suggests exercises tailored to the user’s mood. The HRS to alleviate stress includes recommending books to read [ 54 ] and meditative audios [ 85 ]. The recommender techniques used varied greatly. Table 2 shows the distributions of these recommender techniques. Overview of the different recommender techniques used in the studies.
a The papers are classified based on how the authors reported their techniques. Recommender Techniques in PracticeThe majority of HRSs (49/73, 67%) rely on knowledge-based techniques, either directly (17/49, 35%) or in a hybrid approach (32/49, 65%). Knowledge-based techniques are often used to incorporate additional information of patients into the recommendation process [ 112 ] and have been shown to improve the quality of recommendations while alleviating other drawbacks such as cold-start and sparsity issues [ 14 ]. Some studies use straightforward approaches, such as if-else reasoning based on domain knowledge [ 9 , 79 , 81 , 82 , 88 , 90 , 100 ]. Other studies use more complex algorithms such as particle swarm optimization [ 57 ], fuzzy logic [ 68 ], or reinforcement algorithms [ 44 , 84 ]. In total, 32 studies reported using a combination of recommender techniques and are classified as hybrid recommender systems . Different knowledge-based techniques are often combined. For example, Ali et al [ 56 ] used a combination of rule-based reasoning, case-based reasoning, and preference-based reasoning to recommend personalized physical activities according to the user’s specific needs and personal interests. Asthana et al [ 46 ] combined the knowledge of a decision tree and demographic information to identify the health conditions. When health conditions are known, the system knows which measurements need to be monitored. A total of 7 studies used a content-based technique to recommend educational content [ 15 , 72 , 87 ], activities [ 32 , 86 ], reading materials [ 54 ], or nutritional advice [ 63 ]. Although collaborative filtering is a popular technique [ 113 ], it is not used frequently in the HRS domain. Marlin et al [ 59 ] used collaborative filtering to personalize future smoking cessation messages based on explicit feedback on past messages. This approach is used more often in combination with other techniques. A total of 2 studies [ 38 , 92 ] combined content-based techniques with collaborative filtering. Esteban et al [ 92 ], for instance, switched between content-based and collaborative approaches. The former approach is used for new physiotherapy exercises and the latter, when a new patient is registered or when previous recommendations to a patient are updated. Context-Based Recommender TechniquesFrom an HRS perspective, context is described as an aggregate of various information that describes the setting in which an HRS is deployed, such as the location, the current activity, and the available time of the user. A total of 5 studies use contextual information to improve their recommendations but use a different technique; a prefilter uses contextual information to select or construct the most relevant data for generating recommendations. For example, in Narducci et al [ 75 ], the set of potentially similar patients was restricted to consultation requests in a specific medical area. Rist et al [ 33 ] applied a rule-based contextual prefiltering approach [ 114 ] to filter out inadequate recommendations, for example, “if it is dark outside, all outdoor activities, such as ‘take a walk,’ are filtered out” [ 33 ] before they are fed to the recommendation algorithm. However, a postfilter removes the recommended items after they are generated, such as filtering outdoor activities while it is raining. Casino et al [ 97 ] used a postfiltering technique by running the recommended items through a real-time constraint checker . Finally, contextual modeling, which was used by 2 studies [ 35 , 58 ], uses contextual information directly in the recommendation function as an explicit predictor of a user’s rating for an item [ 114 ]. Location, agenda, and weather are examples of contextual information used by Lin et al [ 35 ] to promote the adoption of a healthy and active lifestyle. Cerón-Rios et al [ 58 ] used a decision tree to analyze user needs, health information, interests, time, location, and lifestyle to promote healthy habits. Casino et al [ 97 ] gathered contextual information through smart city sensor data to recommend healthier routes. Similarly, contextual information was acquired by Rist et al [ 33 ] using sensors embedded in the user’s environment. ComparisonsA total of 8 papers compared different recommender techniques to find the most optimal algorithm for a specific data set, end users, domain, and goal. Halder et al [ 52 ] used two well-known health forum data sets (PatientsLikeMe [ 115 ] and HealthBoards [ 116 ]) to compare 7 recommender techniques (among collaborative filtering and content-based filtering) and found that a hybrid approach scored best [ 52 ]. Another example is the study by Narducci et al [ 75 ], who compared four recommendation algorithms: cosine similarity as a baseline, collaborative filtering, their own HealthNet algorithm, and a hybrid of HealthNet and cosine similarity. They concluded that a prefiltering technique for similar patients in a specific medical area can drastically improve the recommendation accuracy [ 75 ]. The average and SD of the resulting ratings of the two collaborative techniques are compared with random recommendations by Li et al [ 60 ]. They show that a hybrid approach of a collaborative filter augmented with the calculated health level of the user performs better. In their nutrition-based meal recommender system, Yang et al [ 49 ] used item-wise and pairwise image comparisons in a two-step process. In conclusion, the 8 studies showed that recommendations can be improved when the benefits of multiple recommender techniques are combined in a hybrid solution [ 60 ] or contextual filters are applied [ 75 ]. HRSs can be evaluated in multiple ways. In this study, we found two categories of HRS evaluations: (1) offline evaluations that use computational approaches to evaluate the HRS and (2) evaluations in which an end user is involved. Some studies used both, as shown in Multimedia Appendix 3 . Offline EvaluationsOf the total studies, 47% (34/73) do not involve users directly in their method of evaluation. The evaluation metrics also vary greatly, as many distinct metrics are reported in the included papers ( Multimedia Appendix 3 ). Precision 53% (18/34), accuracy 38% (13/34), performance 35% (12/34), and recall 32% (11/34) were the most commonly used offline evaluation metrics. Recall has been used significantly more in recent papers, whereas accuracy also follows an upward trend. Moreover, performance was defined differently across studies. Torrent-Fontbona and Lopez Ibanez [ 81 ] compared the “amount of time in the glycaemic target range by reducing the time below the target” as performance. Cho et al [ 72 ] compared the precision and recall to report the performance. Clarke et al [ 84 ] calculated their own reward function to compare different approaches, and Lin et al [ 35 ] measured system performance as the number of messages sent in their in the wild study. Finally, Marlin et al [ 59 ] tested the predictive performance using a triple cross-validation procedure. Other popular offline evaluation metrics are accuracy-related measurements, such as mean absolute (percentage) error, 18% (6/34); normalized discounted cumulative gain (nDCG), 18% (6/34); F 1 score, 15% (5/34); and root mean square error, 15% (5/34). The other metrics were measured inconsistently. For example, Casino et al [ 97 ] reported that they measure robustness but do not outline what they measure as robustness. However, they measured the mean absolute error. Torrent-Fontbona and Lopez Ibanez [ 81 ] defined robustness as the capability of the system to handle missing values. Effectiveness is also measured with different parameters, such as its ability to take the right classification decisions [ 75 ] or in terms of key opinion leaders’ identification [ 41 ]. Finally, Li and Zaman [ 68 ] measured trust with a proxy: “evaluate the trustworthiness of a particular user in a health care social network based on factors such as role and reputation of the user in the social community” [ 68 ]. User EvaluationsOf the total papers, 53% (39/73) included participants in their HRS evaluation, with an average sample size of 59 (SD 84) participants (excluding the outlier of 8057 participants, as recruited in the study by Cheung et al [ 83 ]). On average, studies ran for more than 2 months (68, SD 56 days) and included all age ranges. There is a trend of increasing sample size and study duration over the years. However, only 17 studies reported the study duration; therefore, these trends were not significant. Surveys (12/39, 31%), user studies (10/39, 26%), and deployments in the wild (10/39, 26%) were the most used user evaluations. Only 6 studies used an RCT to evaluate their HRS. Finally, although all the included studies focused on HRSs and were dealing with sensitive data, only 12% (9/73) [ 9 , 34 , 42 - 45 , 73 , 83 , 95 ] reported ethical approval by a review board. No universal survey was found, as all the studies deployed a distinct survey. Ge et al [ 48 ] used the system usability scale and the framework of Knijnenburg et al [ 117 ] to explain the user experience of recommender systems. Esteban et al [ 95 ] designed their own survey with 10 questions to inquire about user experience. Cerón-Rios [ 58 ] relied on the ISO/IEC (International Organization of Standardization/International Electrotechnical Commission) 25000 standard to select 7 usability metrics to evaluate usability. Although most studies did not explicitly report the surveys used, user experience was a popular evaluation metric, as in the study by Wang et al [ 69 ]. Other metrics range from measuring user satisfaction [ 69 , 99 ] and perceived prediction accuracy [ 59 ] (with 4 self-composed questions). Nurbakova et al [ 98 ] combined data analytics with surveys to map their participants’ psychological background, including orientations to happiness measured using the Peterson scale [ 118 ], personality traits using the Mini-International Personality Item Pool [ 119 ], and Fear of Missing Out based on the Przybylski scale [ 120 ]. Single-Session Evaluations (User Studies)A total of 10 studies recruited users and asked them to perform certain tasks in a single session. Yang et al [ 49 ] performed a 60-person user study to assess its feasibility and effectiveness. Each participant was asked to rate meal recommendations relative to those made using a traditional survey-based approach. In a study by Gutiérrez et al [ 63 ], 15 users were asked to use the health augmented reality assistant and measure the qualities of the recommender system, users’ behavioral intentions, perceived usefulness, and perceived ease of use. Jiang and Xu [ 77 ] performed 30 consultations and invited 10 evaluators majoring in medicine and information systems to obtain an average rating score and nDCG. Radha et al [ 8 ] used comparative questions to evaluate the feasibility. Moreover, Cheng et al [ 89 ] used 2 user studies to rank two degrees of compromise (DOC). A low DOC assigns more weight to the algorithm, and a high DOC assigns more weight to the user’s health perspective. Recommendations with a lower DOC are more efficient for the user’s health, but recommendations with a high DOC could convince users to believe that the recommended action is worth doing. Other approaches used are structured interviews [ 58 ], ranking [ 86 , 89 ], asking for unstructured feedback [ 40 , 88 ], and focus group discussions [ 87 ]. Finally, 3 studies [ 15 , 75 , 90 ] evaluated their system through a heuristic evaluation with expert users. In the WildOnly 2 studies tested their HRS into the wild recruited patients (people with a diagnosed health condition) in their evaluation. Yom-Tov et al [ 44 ] provided 27 sedentary patients with type 2 diabetes with a smartphone-based pedometer and a personal plan for physical activity. They assessed the effectiveness by calculating the amount of activity that the patient performed after the last message was sent. Lima-Medina et al [ 73 ] interviewed 45 patients with cardiovascular problems after a 6-month study period to measure (1) social management results, (2) health care plan results, and (3) recommendation results. Rist et al [ 33 ] performed an in-situ evaluation in an apartment of an older couple and used the data logs to describe the usage but augmented the data with a structured interview. Yang et al [ 49 ] conducted a field study of 227 anonymous users that consisted of a training phase and a testing phase to assess the prediction accuracy. Buhl et al [ 99 ] created three user groups according to the recommender technique used and analyzed log data to compare the response rate, open email rate, and consecutive log-in rate. Similarly, Huang et al [ 76 ] compared the ratio of recommended doctors chosen and reserved by patients with the recommended doctors. Lin et al [ 35 ] asked 6 participants to use their HRSs for 5 weeks, measured system performance, studied user feedback to the recommendations, and concluded with an open-user interview. Finally, Ali et al [ 56 ] asked 10 volunteers to use their weight management systems for a couple of weeks. However, they do not focus on user-centric evaluation, as “only a prototype of the [...] platform is implemented.” Rabbi et al [ 7 ] followed a single case with multiple baseline designs [ 121 ]. Single-case experiments achieve sufficient statistical power with a large number of repeated samples from a single individual. Moreover, Rabbi et al [ 7 ] argued that HRSs suit this requirement “since enough repeated samples can be collected with automated sensing or daily manual logging [ 121 ].” Participants were exposed to 2, 3, or 4 weeks of the control condition. The study ran for 7-9 weeks to compensate for the novelty effects. Food and exercise log data were used to measure changes in food calorie intake and calorie loss during exercise. Randomized Controlled TrialsOnly 6 studies followed an RCT approach. In the RCT by Bidargaddi et al [ 45 ], a large group of patients (n=192) and control group (n=195) were asked to use a web-based recommendation service for 4 weeks that recommended mental health and well-being mobile apps. Changes in well-being were measured using the Mental Health Continuum-Short Form [ 122 ]. The RCT by Sadasivam et al [ 42 ] enrolled 120 current smokers (n=74) and control group (n=46) as a follow-up to a previous RCT [ 123 ] that evaluated their portal to specifically evaluate the HRS algorithm. Message ratings were compared between the intervention and control groups. Cheung et al [ 83 ] measured app loyalty through the number of weekly app sessions over a period of 16 weeks with 8057 users. In the study by Paredes et al [ 34 ], 120 participants had to use the HRS for at least 26 days. Self-reported stress assessment was performed before and after the intervention. Agapito et al [ 67 ] used an RCT with 40 participants to validate the sensitivity (true positive rate/[true positive rate+false negative rate]) and specificity (true negative rate/[true negative rate+false positive rate]) of the DIETOS HRS. Finally, Luo et al [ 93 ] performed a small clinical trial for more than 3 months (but did not report the number of participants). Their primary outcome measures included two standard clinical blood tests: fasting blood glucose and laboratory-measured glycated hemoglobin, before and after the intervention. Only 47% (34/73) of the studies reported implementing a graphical user interface to communicate the recommended health items to the user. As illustrated in Table 3 , 53% (18/34) use a mobile interface, usually through a mobile (web) app, whereas 36% (14/34) use a web interface to show the recommended items. Rist et al [ 33 ] built a kiosk into older adults’ homes, as illustrated in Figure 3 . Gutiérrez et al [ 63 ] used Microsoft HoloLens to project healthy food alternatives in augmented reality surrounding a physical object that the user holds, as shown in Figure 4 . Distribution of the interfaces used among the different health recommender systems (n=34).
 Rist et al installed a kiosk in the home of older adults as a direct interface to their health recommender system.  An example of the recommended healthy alternatives by Gutiérrez et al. VisualizationA total of 7 studies [ 33 , 34 , 37 , 63 , 79 , 88 , 97 ] or approximately one-fourth of the studies with an interface included visualizations. However, the approach used was different for all studies, as shown in Table 4 . Showing stars to show the relevance of a recommended item are only used by Casino et al [ 97 ] and Gutiérrez et al [ 63 ]. Wayman and Madhvanath [ 37 ] also used bar charts to visualize the progress toward a health goal. They visualize the healthy proportions, that is, what the user should eat. Somewhat more complex visualizations are used by Ho and Chen [ 88 ] who visualized the user’s ECG zones. Paredes et al [ 34 ] presented an emotion graph as an input screen. Rist et al [ 33 ] visualized an example of how to perform the recommended activity. Distribution of the visualizations used among the different health recommender systems (n=7).
TransparencyIn the study by Lage et al [ 87 ], participants expressed that: they would like to have more control over recommendations received. In that sense, they suggested more information regarding the reasons why the recommendations are generated and more options to assess them. A total of 7 studies [ 7 , 37 , 41 , 45 , 63 , 66 , 82 ] explained the reasoning behind recommendations to end users at the user interface. Gutiérrez et al [ 63 ] provided recommendations for healthier food products and mentioned that the items ( Figure 4 ) are based on the users’ profile. Ueta et al [ 66 ] explained the relationship between the recommended dishes and a person’s health conditions. For example, a person with acne can see the following text: “15 dishes that contained Pantothenic acid thought to be effective in acne a lot became a hit” [ 66 ]. Li and Kong [ 82 ] showed personalized recommended health actions in a message center. Color codes are used to differentiate between reminders, missed warnings, and recommendations. Rabbi et al [ 7 ] showed tailored motivational messages to explain why activities are recommended. For example, when the activity walk near East Ave is recommended, the app shows the additional message: 1082 walks in 240 days, 20 mins of walk everyday. Each walk nearly 4 min. Let us get 20 mins or more walk here today 7 Wayman and Madhvanath [ 37 ] first visualized the user’s personal nutrition profile and used the lower part of the interface to explain why the item was recommended. They provided an illustrative example of spaghetti squash. The explanation shows that: This product is high in Dietary_fiber, which you could consume more of. Try to get 3 servings a week 37 Guo et al [ 41 ] recommended doctors and showed a horizontal bar chart to visualize the user’s values compared with the average values. Finally, Bidargaddi et al [ 45 ] visualized how the recommended app overlaps with the goal set by the users, as illustrated in Figure 5 .  A screenshot from the health recommender system of Bidargaddi et al. Note the blue tags illustrating how each recommended app matches the users’ goals. Principal FindingsHRSs cover a multitude of subdomains, recommended items, implementation techniques, evaluation designs, and means of communicating the recommended items to the target user. In this systematic review, we clustered the recommended items into four groups: lifestyle, nutrition, general health care information, and specific health conditions. There is a clear trend toward HRSs that provide well-being recommendations but do not directly intervene in the user’s medical status. For example, almost 70% (50/73; lifestyle and nutrition) focused on no strict medical recommendations. In the lifestyle group, physical activities (10/24, 42%) and advice on how to potentially change behavior (7/24, 29%) were recommended most often. In the nutrition group, these recommendations focused on nutritional advice (8/26, 31%), diets (7/26, 27%), and recipes (7/26, 27%). A similar trend was observed in the health care information group, where HRSs focused on guiding users to the appropriate environments such as hospitals (5/23, 22%) and medical professionals (4/23, 17%) or on helping users find qualitative information (5/23, 22%) on validated sources or from experiences by similar users and patients on health care forums (3/23, 13%). Thus, they only provide general information and do not intervene by recommending, for example, changing medication. Finally, when HRSs targeted specific health conditions, they recommended nonintervening actions, such as meditation sessions [ 84 ] or books to read [ 54 ]. Although collaborative filtering is commonly the most used technique in other domains [ 124 ], here only 3 included studies reported the use of a collaborative filtering approach. Moreover, 43% (32/73) of the studies applied a hybrid approach, showing that HRS data sets might need special attention, which might also be the reason why all 73 studies used distinct data sets. In addition, the HRS evaluations varied greatly and were divided over evaluations where the end user was involved and evaluations that did not evolve users (offline evaluations). Only 47% (34/73) of the studies reported implementing a user interface to communicate recommendations to the user, despite the need to show the rationale of recommendations, as echoed by many researchers and practitioners [ 11 ]. Moreover, only 15% (7/47) included a (basic) visualization. Unfortunately, this general lack of agreement on how to report HRSs might introduce researcher bias, as a researcher is currently completely unconstrained in defining what and how to measure the added value of an HRS. Therefore, further debate in the health recommender community is needed on how to define and measure the impact of HRSs. On the basis of our review and contribution to this discussion, we put forward a set of essential information that researchers should report in their studies. Considerations for PracticeThe previously discussed results have direct implications in practice and provide suggestions for future research. Figure 6 shows a reference frame of these requirements that can be used in future studies as a quality assessment tool.  A reference frame to report health recommender system studies. On the basis of the results of this study, we suggest that it should be clear what and how items are recommended (A), who the target user is (B), which data are used (C), and which recommender techniques are applied (D). Finally, the evaluation design should be reported in detail (E). Define the Target UserAs shown in this review, HRSs are used in a plethora of subdomains and each domain has its own experts. For example, in nutrition, the expert is most likely a dietician. However, the user of an HRS is usually a layperson without the knowledge of these domain experts, who often have different viewing preferences [ 125 ]. Furthermore, each user is unique. All individuals have idiosyncratic reasons for why they act, think, behave, and feel in a certain way at a specific stage of their life [ 126 ]. Not everybody is motivated by the same elements. Therefore, it is important to know the target user of the HRS. What is their previous knowledge, what are their goals, and what motivates them to act on a recommended item? Show What Is Recommended (and How)Researchers have become aware that accuracy is not sufficient to increase the effectiveness of a recommender system [ 127 ]. In recent years, research on human factors has gained attention. For example, He et al [ 11 ] surveyed 24 existing interactive recommender systems and compared their transparency, justification, controllability, and diversity. However, none of these 24 papers discussed HRSs. This indicates the gap between HRSs and recommender systems in other fields. Human factors have gained interest in the recommender community by “combining interactive visualization techniques with recommendation techniques to support transparency and controllability of the recommendation process” [ 11 ]. However, in this study, only 10% (7/73) explained the rationale of recommendations and only 10% (7/73) included a visualization to communicate the recommendations to the user. We do not argue that all HRSs should include a visualization or an explanation. However, researchers should pay attention to the delivery of these recommendations. Users need to understand, believe, and trust the recommended items before they can act on it. To compare and assess HRSs, researchers should unambiguously report what the HRS is recommending. After all, typical recommender systems act like a black box , that is, they show suggestions without explaining the provenance of these recommendations [ 11 ]. Although this approach is suitable for typical e-commerce applications that involve little risk, transparency is a core requirement in higher risk application domains such as health [ 128 ]. Users need to understand why a recommendation is made, to assess its value and importance [ 12 ]. Moreover, health information can be cumbersome and not always easy to understand or situate within a specific health condition [ 129 ]. Users need to know whether the recommended item or action is based on a trusted source, tailored to their needs, and actionable [ 130 ]. Report the Data Set UsedAll 73 studies used a distinct data set. Furthermore, some studies combine data from multiple databases, making it even more difficult to judge the quality of the data [ 35 ]. Nonetheless, most studies use self-generated data sets. This makes it difficult to compare and externally validate HRSs. Therefore, we argued that researchers should clarify the data used and potentially share whether these data are publicly available. However, in health data are often highly privacy sensitive and cannot be shared among researchers. Outline the Recommender TechniquesThe results show that there is no panacea for which recommender technique to use. The included studies differ from logic filters to traditional recommender techniques, such as collaborative filtering and content-based filtering to hybrid solutions and self-developed algorithms. However, with 44% (32/73), there is a strong trend toward the use of hybrid recommender techniques. The low number of collaborative filter techniques might be related to the fact that the evaluation sample sizes were also relatively low. Unfortunately, some studies have not fully disclosed the techniques used and only reported on the main algorithm used. It is remarkable that studies published in high-impact journals, such as studies by Bidargaddi et al [ 45 ] and Cheung et al [ 83 ], did not provide information on the recommender technique used. Nonetheless, disclosing the recommender technique allows other researchers not only to build on empirically tested technologies but also to verify whether key variables are included [ 29 ]. User data and behavior data can be identified to augment theory-based studies [ 29 ]. Researchers should prove that the algorithm is capable of recommending valid and trustworthy recommendations to the user based on their available data set. Elaborate on the Evaluation ProtocolsHRSs can be evaluated using different evaluation protocols. However, the protocol should be outlined mainly by the research goals of the authors. On the basis of the papers included in this study, we differentiate between the two approaches. In the first approach, the authors aim to influence their users’ health, for example, by providing personalized diabetes guidelines [ 81 ] or prevention exercises for users with low back pain [ 95 ]. Therefore, the end user should always be involved in both the design and evaluation processes. However, only 8% (6/73) performed an RCT and 14% (10/73) deployed their HRS in the wild. This lack of user involvement has been noted previously by researchers and has been identified as a major challenge in the field [ 27 , 28 ]. Nonetheless, in other domains, such as job recommenders [ 131 ] or agriculture [ 132 ], user-centered design has been proposed as an important methodology in the design and development of tools used by end users, with the purpose of gaining trust and promoting technology acceptance, thereby increasing adoption with end users. Therefore, we recommend that researchers evaluate their HRSs with actual users. A potential model for a user-centric approach to recommender system evaluation is the user-centric framework proposed by Knijnenburg et al [ 117 ]. Research protocols need to be elaborated and approved by an ethical review board to prevent any impact on users. Authors should report how they informed their users and how they safeguarded the privacy of the users. This is in line with the modern journal and conference guidelines. For example, editorial policies of the Journal of Medical Internet Research state that “when reporting experiments on human subjects, authors should indicate IRB (Institutional Rese[a]rch Board, also known as REB) approval/exemption and whether the procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation” [ 133 ]. However, only 12% (9/73) reported their approval by an ethical review board. Acquiring review board approval will help the field mature and transition from small incremental studies to larger studies with representative users to make more reliable and valid findings. In the second approach, the authors aim to design a better algorithm, where better is again defined by the authors. For example, the algorithm might perform faster, be more accurate, and be more efficient in computing power. Although the F 1 score, the mean absolute error, and nDCG are well defined and known within the recommender domain, other parameters are more ambiguous. For example, the performance or effectiveness can be assessed using different measurements. However, a health parameter can be monitored, such as the duration that a user remains within healthy ranges [ 81 ]. Furthermore, it could be a predictive parameter, such as an improved precision and recall as a proxy for performance [ 72 ]. Unfortunately, this difference makes it difficult to compare health recommendation algorithms. Furthermore, this inconsistency in measurement variables makes it infeasible to report in this systematic review which recommender techniques to use. Therefore, we argue that HRS algorithms should always be evaluated for other researchers to validate the results, if needed. LimitationsThis study has some limitations that affect its contribution. Although an extensive scope search was conducted in scientific databases and most relevant health care informatic journals, some relevant literature in other domains might have been excluded. The keywords used in the search string could have impacted the results. First, we did not include domain-specific constructs of health, such as asthma, pregnancy, and iron deficiency. Many studies may implicitly report healthy computer-generated recommendations when they research the impact of a new intervention. In these studies, however, building an HRS is often not their goal and, therefore, was excluded from this study. Second, we searched for papers that reported studying an HRS; nonincluded studies might have built an HRS but did not report it as such. Considering our RQs, we deemed it important that authors explicitly reported their work as a recommender system. To conclude, in this study, we provide a large cross-domain overview of health recommender techniques targeted to laypersons and deliver a set of recommendations that could help the field of HRS mature. This study presents a comprehensive report on the use of HRS across domains. We have discussed the different subdomains HRS applied in, the different recommender techniques used, the different manners in which they are evaluated, and finally, how they present the recommendations to the user. On the basis of this analysis, we have provided research guidelines toward a consistent reporting of HRSs. We found that although most applications are intended to improve users’ well-being, there is a significant opportunity for HRSs to inform and guide users’ health actions. Although many of the studies present a lack of a user-centered evaluation approach, some studies performed full-scale RCT evaluations or elaborated in the wild studies to validate their HRS, showing the field of HRS is slowly maturing. On the basis of this study, we argue that it should always be clear what the HRS is recommending and to whom these recommendations are for. Graphical assets should be added to show how recommendations are presented to users. Authors should also report which data sets and algorithms were used to calculate the recommendations. Finally, detailed evaluation protocols should be reported. We conclude that the results motivate the creation of richer applications in future design and development of HRSs. The field is maturing, and interesting opportunities are being created to inform and guide health actions. AcknowledgmentsThis work was part of the research project PANACEA Gaming Platform with project HBC.2016.0177, which was financed by Flanders Innovation & Entrepreneurship and research project IMPERIUM with research grant G0A3319N from the Research Foundation-Flanders (FWO) and the Slovenian Research Agency grant ARRS-N2-0101. Project partners were BeWell Innovations and the University Hospital of Antwerp. Abbreviations
Multimedia Appendix 1Multimedia appendix 2, multimedia appendix 3. Conflicts of Interest: None declared. Academia.edu no longer supports Internet Explorer. To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser . Enter the email address you signed up with and we'll email you a reset link.
 Movie Recommendation Systems Based on Collaborative Filtering: A Case Study on Netflix Related PapersInternational Journal of Electrical and Computer Engineering (IJECE) Nowadays, recommendation systems are used successfully to provide items (example: movies, music, books, news, images) tailored to user preferences. Amongst the approaches existing to recommend adequate content, we use the collaborative filtering approach of finding the information that satisfies the user by using the reviews of other users. These reviews are stored in matrices that their sizes increase exponentially to predict whether an item is relevant or not. The evaluation shows that these systems provide unsatisfactory recommendations because of what we call the cold start factor. Our objective is to apply a hybrid approach to improve the quality of our recommendation system. The benefit of this approach is the fact that it does not require a new algorithm for calculating the predictions. We are going to apply two algorithms: k-nearest neighbours (KNN) and the matrix factorization algorithm of collaborative filtering which are based on the method of (singular-value-decomposition). Our combined model has a very high precision and the experiments show that our method can achieve better results.  Expressing reviews in the form of sentiments or ratings for item used or movie seen is the part of human habit. These reviews are easily available on different social websites. Based on interest pattern of a user, it is important to recommend him the items. Recommendation system is playing a vital role in everyone's life as demand of recommendation for user's interest increasing day by day. Movie recommendation system based on available ratings for a movie has become interesting part for new users. Till today, a lot many recommendation systems are designed using several machine learning algorithms. Still, sparsity problems, cold start problem, scalability, grey sheep problem are the hurdles for the recommendation systems that must be resolved using hybrid algorithms. We proposed in this paper, a movie rating system using a k-nearest neighbor (KNN-based) collaborative filtering (CF) approach. We compared user's ratings for different movies to get top K users. Then we have used this top K set to find missing ratings by user for a movie using CF. Our proposed system when evaluated for various criteria shows promising results for movie recommendations compared with existing systems. International Journal of Electrical and Computer Engineering (IJECE) , Dajana Conte Due to modern information and communication technologies (ICT), it is increasingly easier to exchange data and have new services available through the internet. However, the amount of data and services available increases the difficulty of finding what one needs. In this context, recommender systems represent the most promising solutions to overcome the problem of the so-called information overload, analyzing users' needs and preferences. Recommender systems (RS) are applied in different sectors with the same goal: to help people make choices based on an analysis of their behavior or users' similar characteristics or interests. This work presents a different approach for predicting ratings within the model-based collaborative filtering, which exploits singular value factorization. In particular, rating forecasts were generated through the characteristics related to users and items without the support of available ratings. The proposed method is evaluated through the MovieLens100K dataset performing an accuracy of 0.766 and 0.951 in terms of mean absolute error and root-mean-square error. International Journal of Emerging Technology and Advanced Engineering Vishal Paranjape The primary aim of recommender system is to predict items which are of most interest to the users and today recommender systems play a vital role in boosting the sales in any e-commerce based platform. The present paper proposes an approach for recommending movies to the users on the basis on their choices. A novel technique for evaluation of collaborative filtering using SVD and hit ratio as a metric is taken in our proposed approach. We attempted to build a model-based Collaborative filtering technique. The proposed paper makes use of matrix factorization techniques like SVD & SVD++ for filtering movie recommendation system based on latent features. It makes better recommendations based on choice of user because it captures the underlying features driving the raw data. In this paper we are proposing a hybrid recommender system fusion of Content Based and SVD to get a new hybrid recommender system. Our proposed model gives the value of RMSE 0.87 for SVD model and RMSE 0.938 for SVD... Nicola Barbieri A New Similarity Measure for User-based Collaborative Filtering in Recommender Systems srikanth T , Shashi Mogalla ABSTRACT Collaborative filtering is a popular approach in recommender Systems that helps users in identifying the items they may like in a wagon of items. Finding similarity among users with the available item ratings so as to predict rating(s) for unseen item(s) based on the preferences of likeminded users for the current user is a challenging problem. Traditional measures like Cosine similarity and Pearson correlation‟s correlation exhibit some drawbacks in similarity calculation. This paper presents a new similarity measure which improves the performance of Recommender System. Experimental results on MovieLens dataset show that our proposed distance measure improves the quality of prediction. We present clustering results as an extension to validate the effectiveness of our proposed method. KEYWORDS Recommender Systems; Collaborative Filtering; Similarity Measure; Cosine Similarity; Pearson Correlation; Clustering; user-based Collaborative Filtering; Cluster Purity; Similarity International Journal of Engineering Applied Sciences and Technology Sonali Suryawanshi Journal of Computer Science IJCSIS Nowadays, with confronted huge volumes of data produced in the world that created this data in organizations, banks, military centers, hospitals, etc. Recommender systems are generated to deal with the problems of huge volumes of data. These systems help to users in many different fields among the massive volume of information to make the right decision. Recommender system by analyzing user behavior suggests to users appropriate services, like electronic stores. Also in today's world, the Internet provides vast amount of data to users. But if not available effective management on aggregate data, these data will be a barrier to progress. Nowadays, with the development of information systems need before than capable of directing users towards goods and services they are desire. But if effective management on aggregate data is not available, these data will be a barrier to progress. Therefore, in this article we offer a new proposed approach by using a hybrid method. To evaluate the proposed approach we used Movie Lens standard dataset. Also we used two techniques of collaborative filtering and content based filtering. For the proposed approach we offer four models. Finally, these models compared by accuracy of prediction and classification error. https://sites.google.com/site/ijcsis/ Ziad Al-Sharif With the advent and explosive growth of the Web over the past decade, recommender systems have become at the heart of the business strategies of e-commerce and Internet-based companies such as Google, YouTube, Facebook, Netflix, LinkedIn, Amazon, etc. Hence, the collaborative filtering recommendation algorithms are highly valuable and play a vital role at the success of such businesses in reaching out to new users and promoting their services and products. With the aim of improving the recommendation performance of such an algorithm, this paper proposes a new collaborative filtering recommendation algorithm based on dimensionality reduction and clustering techniques. The k-means algorithm and Singular Value Decomposition (SVD) are both used to cluster similar users and reduce the dimensionality. It proposes and evaluates an effective two stage recommender system that can generate accurate and highly efficient recommendations. The experimental results show that this new method significantly improves the performance of the recommendation systems . International Journal of Information Technology and Applied Sciences (IJITAS) muhammad sanwal In the current era, a rapid increase in data volume produces redundant information on the internet. This predicts the appropriate items for users a great challenge in information systems. As a result, recommender systems have emerged in this decade to resolve such problems. Various e-commerce platforms such as Amazon and Netflix prefer using some decent systems to recommend their items to users. In literature, multiple methods such as matrix factorization and collaborative filtering exist and have been implemented for a long time, however recent studies show that some other approaches, especially using artificial neural networks, have promising improvements in this area of research. In this research, we propose a new hybrid recommender system that results in better performance. In the proposed system, the users are divided into two main categories, namely average users, and non-average users. Then, various machine learning and deep learning methods are applied within these categorie... Loading Preview Sorry, preview is currently unavailable. You can download the paper by clicking the button above. RELATED PAPERSComputational Intelligence and Neuroscience Fawaz Alarfaj International Journal of Scientific Research in Computer Science, Engineering and Information Technology International Journal of Scientific Research in Computer Science, Engineering and Information Technology IJSRCSEIT IRJET Journal IEEE Access Nima Joorabloo Paulina Rosa Information Sciences Juan C Burguillo Knowledge-Based Systems EAI Endorsed Transactions on Creative Technologies Sonika Malik Ali Roumani Zeinab Sharifi Saransh Agarwal abdelkader grota Nancy Naina JURNAL MEDIA INFORMATIKA BUDIDARMA ERWIN BUDI SETIAWAN Divya Kumawat International Journal of Engineering & Technology Rajesh Ojha(CSE DEPT) The Ijes The Ijes International Journal of Computational Intelligence Systems (IJCIS), Volume 16, Issue 1 Loc Nguyen's Academic Network , Loc Nguyen RELATED TOPICS
 Recommendation engineRecostream is an AI/ML personalized recommendation engine offered as a SaaS service for online stores of any size. The software is built on the foundation of the Openkoda, utilizing a significant number of pre-existing features within the platform.  Recostream is a product recommendation engine built with Openkoda
 Key features of the application
The ambitious scope of the product and the fast-paced nature of the ecommerce industry required to:
Ideally, the implementation should focus on research and development of the recommendation engine’s machine learning algorithms, with the rest of the system delivered with minimal development effort, if possible.  Recostream’s engineering team decided to use Openkoda as the foundation for the recommendation service, as a significant number of ready-to-use Openkoda features were already available in the platform. The team started with a smaller PoC focused on building the prototype of the recommendation engine only, exploring and testing different machine learning algorithms and scaling the research prototypes to handle real, massive amounts of collected data describing shopping behavior and user interactions. Once the system was validated and promoted from the experimental phase to production, the same team of developers was able to take over Openkoda and build a full enterprise SaaS service around the engine, reusing a large number of features, components, and integrations that already existed in Openkoda. The engineering team of only 4 developers was able to evolve the bare prototype of the recommendation engine into a working product prototype and start the first implementations within two months after the PoC phase was completed. Such a short delivery time was made possible by adding their highly specialized IP behind the developed AI/ML recommendation engine to an enterprise-ready SaaS platform provided by Openkoda. Since the Openkoda platform is freely available as an open-source system under the MIT license , there were no licensing costs associated with using Openkoda in the project.  Recostream quickly became an industry-recognized product recommendation service for ecommerce stores of all sizes and was acquired two years later by GetResponse , a global leader in marketing automation. Read more about the product history and the acquisition . The Recostream product is now a part of the larger offering from GetResponse. What is Openkoda?Openkoda enables faster application development. This AI-powered open source platform provides pre-built application templates and customizable solutions to help you quickly build any functionality you need to run your business. With Openkoda, you can achieve up to 60% faster development and significant cost savings. Start streamlining your development process with Openkoda today. Reduce time-to-marketBuild enterprise applications based on openkoda, loved by developers.  Build business applications fasterOpen-source business application platform for fast development Laciarska 4 50-140 Wroclaw, Poland [email protected]
 Article ContentsIntroduction, methods and evaluation, conclusion and future work, acknowledgement, conflicts of interest..
A content-based dataset recommendation system for researchers—a case study on Gene Expression Omnibus (GEO) repository
Braja Gopal Patra, Kirk Roberts, Hulin Wu, A content-based dataset recommendation system for researchers—a case study on Gene Expression Omnibus (GEO) repository, Database , Volume 2020, 2020, baaa064, https://doi.org/10.1093/database/baaa064
It is a growing trend among researchers to make their data publicly available for experimental reproducibility and data reusability. Sharing data with fellow researchers helps in increasing the visibility of the work. On the other hand, there are researchers who are inhibited by the lack of data resources. To overcome this challenge, many repositories and knowledge bases have been established to date to ease data sharing. Further, in the past two decades, there has been an exponential increase in the number of datasets added to these dataset repositories. However, most of these repositories are domain-specific, and none of them can recommend datasets to researchers/users. Naturally, it is challenging for a researcher to keep track of all the relevant repositories for potential use. Thus, a dataset recommender system that recommends datasets to a researcher based on previous publications can enhance their productivity and expedite further research. This work adopts an information retrieval (IR) paradigm for dataset recommendation. We hypothesize that two fundamental differences exist between dataset recommendation and PubMed-style biomedical IR beyond the corpus. First, instead of keywords, the query is the researcher, embodied by his or her publications. Second, to filter the relevant datasets from non-relevant ones, researchers are better represented by a set of interests, as opposed to the entire body of their research. This second approach is implemented using a non-parametric clustering technique. These clusters are used to recommend datasets for each researcher using the cosine similarity between the vector representations of publication clusters and datasets. The maximum normalized discounted cumulative gain at 10 (NDCG@10), precision at 10 (p@10) partial and p@10 strict of 0.89, 0.78 and 0.61, respectively, were obtained using the proposed method after manual evaluation by five researchers. As per the best of our knowledge, this is the first study of its kind on content-based dataset recommendation. We hope that this system will further promote data sharing, offset the researchers’ workload in identifying the right dataset and increase the reusability of biomedical datasets. Database URL : http://genestudy.org/recommends/#/ In the Big Data era, extensive amounts of data have been generated for scientific discoveries. However, storing, accessing, analyzing and sharing a vast amount of data are becoming major bottlenecks for scientific research. Furthermore, making a large number of public scientific data findable, accessible, interoperable and reusable is a challenging task. The research community has devoted substantial effort to enable data sharing. Promoting existing datasets for reuse is a major initiative that gained momentum in the past decade ( 1 ). Many repositories and knowledge bases have been established for specific types of data and domains. Gene Expression Omnibus (GEO) (https://www.ncbi.nlm.nih.gov/geo/), UKBioBank (https://www.ukbiobank.ac.uk/), ImmPort (https://www.immport.org/shared/home) and TCGA (https://portal.gdc.cancer.gov/) are some examples of repositories for biomedical datasets. DATA.GOV archives the U.S. Government’s open data related to agriculture, climate, education, etc. for research use. However, a researcher looking for previous datasets on a topic still has to painstakingly visit all the individual repositories to find relevant datasets. This is a tedious and time-consuming process. An initiative was taken by the developers of DataMed (https://datamed.org) to solve the aforementioned issues for the biomedical community by combining biomedical repositories together and enhancing the query searching based on advanced natural language processing (NLP) techniques ( 1 , 2 ). DataMed indexes provides the functionality to search diverse categories of biomedical datasets ( 1 ). The research focus of this last work was retrieving datasets using a focused query. In addition to that biomedical and healthCAre Data Discovery Index Ecosystem (bioCADDIE) dataset retrieval challenge was organized in 2016 to evaluate the effectiveness of information retrieval (IR) techniques in identifying relevant biomedical datasets in DataMed ( 3 ). Among the teams participated in this shared task, use of probabilistic or machine learning based IR ( 4 ), medical subject headings (MeSH) term based query expansion ( 5 ), word embeddings and identifying named entity ( 6 ), and re-ranking ( 7 ) for searching datasets using a query were the prevalent approaches. Similarly, a specialized search engine named Omicseq was developed for retrieving omics data ( 8 ). Google Dataset Search (https://toolbox.google.com/datasetsearch) provides the facility to search datasets on the web, similar to DataMed. While DataMed indexes only biomedical domain data, indexing in Google Dataset Search covers data across several domains. Datasets are created and added to repositories frequently, which makes it difficult for a researcher to know and keep track of all datasets. Further, search engines such as DataMed or Google Dataset Search are helpful when the user knows what type of dataset to search for, but determining the user intent in web searches is a difficult problem due to the sparse data available concerning the searcher ( 9 ). To overcome the aforementioned problems and make dataset search more user-friendly, a dataset recommendation system based on a researcher’s profile is proposed here. The publications of researchers indicate their academic interest, and this information can be used to recommend datasets. Recommending a dataset to an appropriate researcher is a new field of research. There are many datasets available that may be useful to certain researchers for further exploration, and this important aspect of dataset recommendation has not been explored earlier. Recommendation systems, or recommenders, are an information filtering system that deploys data mining and analytics of users’ behaviors, including preferences and activities, for predictions of users’ interests on information, products or services. Research publications in recommendation systems can be broadly grouped as content-based or collaborative filtering recommendation systems ( 10 ). This article describes the development of a recommendation system for scholarly use. In general, developing a scholarly recommendation system is both challenging and unique because semantic information plays an important role in this context, as inputs such as title, abstract and keywords need to be considered ( 11 ). The usefulness of similar research article recommendation systems has been established by the acceptance of applications such as Google Scholar (https://scholar.google.com/), Academia.edu (https://www.academia.edu/), ResearchGate (https://www.researchgate.net/), Semantic Scholar (https://www.semanticscholar.org/) and PubMed (https://www.ncbi.nlm.nih.gov/pubmed/) by the research community. Dataset recommendation is a challenging task due to the following reasons. First, while standardized formats for dataset metadata exist ( 12 ), no such standard has achieved universal adoption, and researchers use their own convention to describe their datasets. Further, many datasets do not have proper metadata, which makes the prepared dataset difficult to reuse/recommend. Second, there are many dataset repositories with the same dataset in different formats, making recommendation a challenging task. Additionally, the dataset recommendation system should be scalable to the increasing number of online datasets. We cast the problem of recommending datasets to researchers as a ranking problem of datasets matched against the researcher’s individual publication(s). The recommendation system can be viewed as an IR system where the most similar datasets can be retrieved for a researcher using his/her publications. Data linking or identifying/clustering similar datasets have received relatively less attention in research on recommendation systems. Previous work on this topic includes ( 13–15 ). Reference ( 13 ) defined dataset recommendation as to the problem of computing a rank score for each of a set of target datasets ( D T ) so that the rank score indicates the relatedness of D T to a given source dataset ( D S ). The rank scores provide information on the likelihood of a D T to contain linking candidates for D S . Reference ( 15 ) proposed a dataset recommendation system by first creating similarity-based dataset networks, and then recommending connected datasets to users for each dataset searched. Despite the promising result this approach suffers from the cold start problem. Here cold start problem refers to the user’s initial dataset selection, where the user has no idea what dataset to select/search. If a user chooses a wrong dataset initially, then the system will always recommend wrong datasets to the user. Some experiments were performed to identify datasets shared in the biomedical literature ( 16–18 ). Reference ( 17 ) identified data shared in biomedical literature articles using regular expression patterns and machine learning algorithms. Reference ( 16 ) identified datasets in social sciences papers using a semi-automatic method. The last system reportedly performed well (F-measure of 0.83) in finding datasets in the da|ra dataset registry. Different deep learning methods were used to extract the dataset mentions in publication and detect mention text fragment to a particular dataset in the knowledge base ( 18 ). Further, a content-based recommendation system was developed for recommending literature for datasets in ( 11 ), which was the first step toward developing a literature recommendation tool by recommending relevant literature for datasets. This article proposes a dataset recommender that recommends datasets to researchers based on their publications. We collected dataset metadata (title and summary) from GEO and researcher’s publications (title, abstract and year of publication) from PubMed using name and curriculum vitae (CV) for developing a dataset recommendation system. A vector space model (VSM) is used to compare publications and datasets. We propose two novel ideas: A method for representing researchers with multiple vectors reflecting each researcher’s diverse interests. A system for recommending datasets to researchers based on their research vectors. For the datasets, we focus on GEO (https://www.ncbi.nlm.nih.gov/geo/). GEO is a public repository for high-throughput microarray and next-generation sequence functional genomics data. It was found that an average of 21 datasets was added daily in the last 6 years (i.e. 2014–19). This gives a glimpse of the increasing number of datasets being made available online, considering that there are many other online data repositories as well. Many of these datasets were collected at significant expense, and most of these datasets were used only once. We believe that reusability of these datasets can be improved by recommending these to appropriate researchers. Efforts on restructuring GEO have been performed by curating available metadata. In reference ( 19 ), the authors identified the important keywords present in the datasets descriptions and searched other similar datasets. Another task on restructuring the GEO database, ReGEO (http://regeo.org/) was developed by ( 20 ), who identified important metadata such as time points and cell lines for datasets using automated NLP techniques. We developed this dataset recommendation system for researchers as a part of the dataset reusability platform (GETc Research Platform(http://genestudy.org/)) for GEO developed at the University Texas Health Science Center at Houston. This website recommends datasets to users using their publications. The rest of the article is organized in the following manner. Section 2 provides an overview of GEO datasets and researcher publications. Methods used for developing the recommendation system and evaluation techniques used in this experiment are described in Section 3 . Section 4 describes results. Section 5 provides a discussion. Finally, conclusion and future directions are discussed in Section 6 . The proposed dataset recommendation system requires both dataset metadata and the user profile for which datasets will be recommended. We collected metadata of datasets from the GEO repository, and researcher publications from PubMed using their names and CVs. The data collection methods and summaries of data are discussed next. GEO DatasetsGEO is one of the most popular public repositories for functional genomics data. As of December 18, 2019, there were 122 222 series of datasets available in GEO. Histograms of datasets submitted to GEO per day and per year as presented in Figure 1 showed an increasing trend of submitting datasets to GEO, which justified our selection of this repository for developing the recommendation system.  Histogram of datasets submitted to GEO based on datasets collected on December 18, 2019  Overview of dataset indexing pipeline Statistics of datasets collected from GEO
For the present experiment, metadata such as title, summary, submission date and name of dataset creator(s) were collected from GEO and indexed in a database, as shown in Figure 2 . We also collected the PMIDs of articles associated with each dataset. However, many datasets did not have articles associated with them. The detailed information of collected datasets is presented in Table 1 . Out of a total of 122 222 GEO datasets, 89 533 had 92 884 associated articles, out of which 61 228 were unique. The maximum number of articles associated with the datasets (‘GSE15907’ and ‘GSE31312’) was 10. These articles were used to remove the publications that were not related to GEO. Further, we used the GEO-related publications for building word embeddings to be used for subsequent text normalization as outlined in Section 3 . Researcher publicationsA researcher’s academic interest can be extracted from publications, grants, talks, seminars and much more. All this information is typically available in the CV, but it is presented in the form of titles/short texts. Here, short texts imply limited information. Further, lack of standardization in CV formats poses challenges to parse the CVs. In this work, an alternative approach was undertaken, which is outlined next. Title and year of the researcher’s publications were present in the CV. However, we required title, abstract and year of publication for our experiment. A researcher’s list of publications (titles and abstracts) are easier to get from web sources such as Google Scholar, PubMed, Semantic Scholar and others. Unfortunately, the full texts of most scientific articles are not publicly available. Thus, for the present experiment, we used only the title and abstract of publications in identifying the researcher’s areas of research.  Overview of researcher’s publication extraction system to remove the author disambiguation Given a researcher, we searched the researcher’s name in PubMed using Entrez API (https://www.ncbi.nlm.nih.gov/books/NBK25 501/) and collected all the publications. Multiple researchers with exact same name might exist, thus, querying the name in PubMed might sometime result in publications from other researchers as well. This is a typical challenge of author disambiguation. However, there are a few attempts that have been undertaken to resolve the issue of author disambiguation, and one of them is ORCID (https://orcid.org). A researcher needs to provide ORCID id to access his/her ORCID details. However, to the best of our knowledge, many researchers in the biomedical domain did not have an associated ORCID account. Thus we used a simple method to disambiguate the authors by using their CVs. Initially, the recommendation system prompts a researcher to provide his/her name and a CV (or list of publications). Next, we collected the publications (titles, names, MeSH terms and year of publication) for a researcher from PubMed by searching his/her name. For removing the publications of other authors with the same name, titles of all collected publications from PubMed were matched against the titles present in the CV. In the case of a match, publications were kept for further processing. An overview of the technique used for the researcher’s publication collection is provided in Figure 3 . One of the limitations of the above publication collection method is that the publications could not be collected if they were not listed in PubMed. Further, the datasets used in the present experiments were from the biomedical domain, and the publications not listed in PubMed were less pertinent to biomedical datasets. For example, someone’s biomedical interests (in PubMed) may be more reliable markers for biomedical datasets than a theoretical computer science or statistics paper. Another downside is if the researcher’s CV may not be fully up-to-date. This section describes how the two main objects of interest (datasets and publications of researchers) were embedded in a vector space and then how these vectors were compared in order to make recommendations. First, both datasets and papers were treated as text objects: the text of a dataset includes its title and summary, while the text of a paper includes its title and abstract. Pre-processing was performed on both a researcher’s publications and datasets by removing the low-value stopwords, links, punctuation and junk words. Further, the nltk WordNet lemmatizer (https://www.nltk.org/_modules/nltk/stem/wordnet.html) was used to get the root forms of the words. Next, we describe the methods used for converting datasets and researchers into vectors. Dataset vector generationVSMs can be built from text in a variety of ways, each of which has its distinct advantages and thus merit experimentation. For the present experiment, we used TF-IDF because it achieved better results for related literature recommendation for datasets in ( 11 ). TF-IDF : For vocabulary W , each unique word w ∈ W is assigned a score proportional to its frequency in the text (term frequency, TF) and its inverse frequency in the full collection (inverse document frequency, IDF). We tuned parameters such as minimum document frequency (min-df) and maximum n-gram size. For the present study, we kept maximum n-gram size = 2 (i.e. unigrams and bigrams) as including the higher n-gram increases the sparsity as well as computational complexity. We converted each dataset into a vector using TF-IDF. For each dataset, the title and summary were preprocessed and normalized and then converted into a single vector. Finally, each publication vector (or publication cluster vector) is compared with dataset vectors to generate the recommendation score. Different methods for representing a researcher’s papers as vectors are discussed next. Researcher vector generationBaseline method. For the baseline method, we combined multiple text-derived paper vectors into a single researcher vector ( v r ) in the same vector space using Equation ( 1 ): where P r is the set of papers of a researcher r ; N r is the total number of papers of that researcher, and it acts as a normalization term; v p is the vector for a single paper p using TF-IDF; λ p is a recency penalty to favor more recent papers (thus better reflecting the researcher’s current interest). It is evident that a researcher will be interested in datasets recommended for his/her current work rather than the work performed a few years back. Thus, we penalized each of the paper vectors from a different year, as stated in Equation ( 2 ): where t is the difference between the current year and year of publication. k is the decaying function to decrease the rate proportional to its current value, and for the present study, we kept k =0.05. . Multi-interest dataset recommendation (MIDR)The baseline method for creating a researcher vector may be helpful for new researchers without many publications, whereas an established researcher may have multiple areas of expertise with multiple papers in each. Also, if the number of papers is imbalanced in multiple areas, then the above baseline method may not work. With a highly imbalanced set of publications this would obviously bias dataset recommendation to the dominant interest. For a more balanced set of interests that are highly dispersed, this mixture would result in the ‘centroid’ of these interests, which could be quite distinct from the individual interests. Both these cases are undesirable. The centroid of a researcher’s interests may not be of much interest to them (e.g. a researcher interested in mouse genomics and HIV vaccines may not be interested in mouse vaccines ). For example, initial experiments were performed on Researcher 1 (mentioned later in Section 4), and it was observed that the datasets recommended for a researcher were biased toward a single research area with the largest number of publications. For example, Researcher 1 has a dominant number of publications on HIV and the baseline system recommends only HIV datasets, even if Researcher 1 has multiple research areas. A critical limitation of the above baseline approach is that researchers can have multiple areas of expertise. We can easily build multiple vectors, each corresponding to a different expertise if we know how to properly group/cluster a researcher’s papers according to expertise or topic. However, parametric methods such as k-means clustering and latent Dirichlet allocation require specifying a priori how many clusters/topics to utilize. Generalizing the number of clusters is not possible due to a varying number of publications of researchers. Instead, our insight is that the more publications a researcher has, the more interests or areas of expertise he/she likely has as well, but this should be modeled as a ‘soft’ constraint rather than a ‘hard’ constraint. We propose to employ the non-parametric Dirichlet Process Mixture Model (DPMM) ( 21 ) to cluster papers into several groups of expertise.  High level architecture of proposed dataset recommendation DPMM : We employed a Gibbs Sampling-based Dirichlet Process Mixture Modeling for text clustering. DPMM offers the following advantages over its traditional counterparts. First, the number of text clusters need not be specified; second, it is relatively scalable and third, it is robust to outliers ( 22 ). The technique employs a collapsed Gibbs Sampling mechanism for Dirichlet process models wherein the clusters are added or removed based on the probability of a cluster associating with a given document. The scalability of the technique stems from the fact that word frequencies are used for text clustering. This reduces the computational burden significantly, considering the large number of samples associated with text processing problems. Further, the optimal number of clusters is likely to be chosen, as clusters with low association probability with documents are eliminated, and new clusters are created for documents that do not belong to selected clusters with high probability. For example, if a cluster c 1 contains five documents, each with low association probability, then the cluster c 1 is eliminated, and new clusters are initialized. In DPMM, the decision to create a new cluster is based on the number of papers to be clustered and the similarity of a given paper to previously clustered papers. Thus, researchers with many papers but few interests can still result in fewer clusters than a researcher with fewer papers but more interests. For example, our evaluation includes two researchers, one with 53 papers and one with 32; however, the DPMM resulted in five and six clusters, respectively. After clustering, we created a pseudo-researcher for each cluster using Equation ( 1 ), though one that can be tied back to the original researcher. The recommendation system uses these pseudo-researchers in its similarity calculations along the same lines as described above. Further, the α parameter was tuned to control the number of clusters ( 22 ). We describe tuning of the α parameter in Section 3.4. Text normalization : Text normalization plays an important role in improving the performance of any NLP system. We also implemented text normalization techniques to improve the efficiency of the proposed clustering algorithm. We normalized similar words by grouping them together and replacing them with the most frequent words in the same word group. For example, HIV, HIV-1, HIV/AIDS and AIDS were replaced with the most frequent word HIV . For identifying similar words, we trained a word2vec model on the articles from PubMed using Gensim (https://radimrehurek.com/gensim/). The datasets are related to gene expressions, while the articles collected from PubMed contain a variety of topics related to biomedicine and life sciences which may not be suitable for building a word embedding in the current study (since some of these articles are highly unrelated to the type of information in GEO). The articles before 1998 were removed as the research on micro-array data started during that year ( 23 ). The publications related to GEO are filtered using the MeSH terms. We also developed a MeSH term classification system for those publications without MeSH terms. More details on GEO related publications filtering can be found in ( 11 ). The similar words were identified using the most_similar function of word2vec . We only considered the top five similar words for each word using most_similar function. The normalized text was used for clustering. It was observed from the initial experiments that the text normalization improved clustering and resulted in the reduced number of clusters using DPMM. Dataset recommendationThe most similar datasets can be recommended to researchers simply by comparing the cosine similarity of the researcher and dataset vectors using Equation ( 3 ): where D is all the datasets that can be recommended to researcher r ; |$\cos(v_r, v_d)$| is the cosine similarity between researcher vector ( v r ) and dataset vector ( v d ). The high-level system architecture of the dataset recommendation system is shown in Figure 4 . This dataset recommendation system is initiated by a researcher (user) by submitting his/her name and CV (or list of publications). The name is searched in PubMed for publication details, and then titles of publications from PubMed were matched with publication titles in CV. The matched publications are then clustered using DPMM to identify research fields of the researcher. Finally, the top similar datasets are recommended using the calculated cosine similarity between the researcher vector (or researcher’s cluster vector) and dataset vectors. The researcher vector (or researcher’s cluster vector) is calculated using Equation ( 1 ). Three dataset recommendation systems are evaluated in this article: a baseline method using the researcher’s vector generation method and two proposed methods using the proposed researcher’s vector generation method. Baseline systemThe baseline system uses the researcher’s vector using Equation ( 1 ) of the baseline method in Section 3.2.1 . The top datasets are recommended after calculating the cosine similarity between the researcher’s vector and dataset vectors. This system reflects only one research field for each researcher. MIDR SystemThe cluster vectors are generated using the modified Equation ( 1 ). Here, cluster-specific research area vectors are created for each researcher, instead of a single vector for each researcher as in baseline system. Papers in a single cluster are multiplied with their recency factors and summed. Then, the summation was divided the number of papers in that cluster. This system uses multiple pseudo vectors for multiple clusters of a researcher ( |$v_{c_i}$| for i th cluster), indicating different research fields that a researcher might have, as mentioned in Section 3.2.2 . This system compares each cluster vector with the dataset vectors and recommends the top datasets by computing the cosine similarity among them. Finally, it merges all the recommended datasets in a round-robin fashion for all the clusters, so that the researcher is able to see various datasets related to different research fields together. MIDR System (Separate)This system is an extension of our proposed MIDR system. Some researchers liked the way recommended datasets were merged. However, other researchers wanted dataset recommendations for each cluster separately. For this reason, another system was developed where the recommended datasets were shown separately for each research cluster, allowing researchers to obtain different recommended datasets for different research interests. Number of clusters with varying α values for proposed α based on our initial evaluation. Abbreviations: P: Proposed, a: total number of clusters, b: number of clusters which contains more than one paper, c: number of clusters which contains only one paper
Tuning the α parameterA researcher with a higher number of publications is more likely to have more research interests. In this paper, research interests are represented as clusters, expressed as vectors. A Dirichlet process is non-parametric because, in theory, there can be an infinite number of clusters. By changing the α parameter, DPMM can vary the number of clusters. The α value is inversely related to the number of clusters, i.e. decreasing the α parameter in DPMM may increase the number of output clusters. Therefore, we propose an α value, which is also inversely related to the number of research publications. Further, the α value must stabilize after a certain threshold to avoid the formation of too many clusters, and it must be generalized to the number of publications. To this end, α is calculated as follows: where N is the total number of papers for a researcher. The α value is proposed based on manually observing the clusters and collecting feedback from different researchers. Apart from inherent requirements for setting α , Equation ( 4 ) maintains a reasonable number of clusters, which was found useful by most of the evaluators. Different α values and their corresponding number of clusters are provided in Table 2 . The number of clusters are divided into three categories: (a) total number of clusters, (b) number of clusters which contains more than one paper, (c) number of clusters which contains only one paper. We removed the clusters with one paper and used the clusters with two or more papers for recommending datasets. We observed that the number of clusters did not entirely depend upon the number of papers, a researcher had. Moreover, it largely reflected the number of research fields that the researcher participated in. For example, Researcher 2 had fewer publications than Researcher 1 and Researcher 3, but the number of clusters was more than the others. This shows that non-parametric clustering is a good technique for segmenting research areas. There is no existing labeled clustered publication datasets available for automatic evaluation. Again, manually evaluating the clusters was a time and resource-consuming task. It might be biased as the evaluation depends upon different judgments for different researchers. Thus, we implemented K-Means for comparing to the proposed DPMM. The automatic cluster comparison was performed using inter- and intra-cluster cosine similarity (IACCS) of words and MeSH terms in the publications, separately. IACCS was the mean cosine similarity of words or MeSH terms for each pair of papers in a given cluster. Considering a cluster of size n ( |$X=\{x_1, x_2, \dots x_n\}$| ), the IACCS can be formulated using Equation ( 5 ): where, x i and x j are the list of MeSH terms or words of the i th and j th paper, respectively, and |$\cos(x_i, x_j)$| is the cosine similarity between them. Finally, the mean of IACCS was calculated using the IACCS of individual clusters. We computed the mean cosine similarity between words or MeSH terms of papers within clusters to calculate the inter-cluster cosine similarity (ICCS). Considering n clusters ( |$c_1, c_2, \dots c_n$| ), ICSS can be formulated using Equation ( 6 ): where, c i and c j are the list of MeSH terms or words of all the papers in the i th and j th clusters, respectively, and |$\cos(c_i, c_j)$| is the cosine similarity between them. For the baseline comparison, publication vectors are created using TF-IDF, then K-Means is used to compute the publication clusters. K-Means is a parametric unsupervised clustering. We implemented K-Means with two and five clusters separately for comparison purposes. On the other hand, the tuning parameter proposed for DPMM resulted in a variable number of clusters for different researchers, and these clusters were used for comparison. Recommendation systemMean IACCS and ICCS for K-Means and DPMM (with different cluster sizes as mentioned in Table 2 ).
Being a novel task, no prior ground truth annotations exist for publication-driven dataset recommendation. Thus, we performed a manual evaluation for each developed dataset recommendation system. We asked researchers to rate each retrieved dataset based on their publications or publication clusters. The researchers included in this study have already worked on the datasets from GEO and published papers on these datasets. The rating criterion was how likely they want to work on the retrieved datasets. We asked them to rate using one to three ‘stars’, with three stars being the highest score. Later, normalized discounted cumulative gain (NDCG) at 10 and Precision at 10 (P@10) were calculated to evaluate different systems. The ratings are: 1 star [not relevant] : This dataset is not useful at all. 2 star [partially relevant] : This dataset is partially relevant to the publication cluster. The researcher has already used this dataset or maybe work on it in the future. 3 star [most relevant] : This dataset is most relevant to the publication cluster, and the researcher wants to work on this dataset as soon as possible. The primary evaluation metric used in this work is NDCG, which is a family of ranking measures widely used in IR applications. It has advantages compared to many other measures. First, NDCG allows each retrieved document to have a graded relevance, while most traditional ranking measures only allow binary relevance (i.e. each document is viewed as either relevant or not relevant). This enables the three-point scale to be directly incorporated into the evaluation metric. Second, NDCG involves a discount function over the rank while many other measures uniformly weight all positions. This feature is particularly important for search engines as users care about top-ranked documents much more than others ( 24 ). NDCG is calculated as follows: where rating ( i ) is the i th dataset rating provided by users. For the present study, we set p = 10 for the simplicity of manual annotation. The NDCG@10 for the baseline and MIDR systems is calculated using the ratings of only the top ten retrieved datasets. For the MIDR system (separate), there were multiple publication clusters for a single user, and for each publication cluster we recommended datasets separately. NDCG@10 was calculated for each publication cluster using the top ten datasets and later averaged to get a final NDCG@10 for a single researcher. For NDCG@10 calculation, the 1-star, 2-star and 3-star are converted to 0, 1 and 2, respectively. We also calculated P@10 (strict and partial) for the baseline and proposed systems. Strict considers only 3-star, while partial considers both 2- and 3-star results. The results presented in this study were evaluated using a total of five researchers (with an average of 32 publications) who already worked on GEO datasets. This is admittedly a small sample size, but is large enough to draw coarse comparisons on this novel task. We compared DPMM clustering with K-Means as mentioned in Section 3.5 . ICCS and mean IACCS values for different clustering methods are presented in Table 3 . In general, higher mean IACSS and lower ICCS generally indicate better clustering. However, this is not always the case, especially when the number of clusters are small, and each cluster contains multiple publications for a single researcher. In this situation, the IACSS for individual cluster decreases after being divided by the number of publication pairs in each cluster. Furthermore, DPMM and K-Means were comparable when the number of clusters produced by both were close to each other. For all the cases, DPMM had higher mean IACSS and lower ICCS than K-Means using words. This suggests that DPMM was well-suited for clustering a researcher’s publications into multiple research fields. Researcher-specific results of the dataset recommendation system are shown in Table 4 . The results for individual researchers are listed for all the systems. Metric-specific average results for all the systems are also shown in Table 4 . The baseline system did not have any publication clusters and all publications were vectorized using Equation ( 1 ). Next, the top ten similar datasets were used to evaluate the results of the baseline system and it obtained the average NDGC@10, P@10 (P) and P@10 (S) of 0.80, 0.69 and 0.45, respectively. The proposed MIDR system obtained the average NDCG@10, P@10 (P) and P@10 (S) of 0.89, 0.78 and 0.61, respectively. The proposed MIDR (separate) system obtained the average NDGC@10, P@10 (P) and P@10 (S) of 0.62, 0.45 and 0.31, respectively. For calculating NDCG@10 and P@10 in the proposed MIDR (separate) system, individual cluster scores were calculated first, and then divided by the total number of clusters. NDCG@10, partial and strict P@10 values of the different dataset recommendation systems based on three evaluators. Abbreviations: Partial: P; Strict: S
The proposed MIDR system performed better than the baseline system. The MIDR system recommended a variety of datasets involving multiple clusters/research fields as opposed to the baseline system recommended datasets from a single research field with the maximum number of publications. Performances of the baseline and proposed MIDR (separate) systems could not be directly compared. Evaluation of the MIDR (separate) system was performed over multiple clusters with ten datasets recommended for each cluster. In contrast, evaluation of the baseline system was performed only on 10 datasets, for example, for Researcher 1 in Table 4 , evaluations of baseline system and MIDR (separate) system were performed on 10 and 50 datasets, respectively. There were other advantages of the MIDR (separate) system over the baseline system, irrespective of higher NDCG@10 for the latter. The baseline system had a bias toward a specific research field which was eliminated in the MIDR (separate) system. For Researchers 1 and 2 in Table 4 , the datasets recommended by the baseline system were found in the results of two clusters/research fields (which had the maximum number of publications) in the proposed MIDR (separate) system. However, for Researcher 3 in Table 4 , recommended datasets of the baseline system were found in the results of only one research field (with the maximum number of publications) in the proposed MIDR (separate) system. For Researcher 1 in Table 4 , there were 31 papers with HIV keywords and those papers were not published recently. We penalized the papers according to the year of publication for all methods. However, the top datasets contained ‘HIV’ or related keywords for the baseline method. We manually checked the top 100 results and found that those were relevant to HIV . Whereas, the proposed MIDR system clustered the publications into different groups (such as HIV, Flu/Influenza , and others), which resulted in recommendations for different research fields. Therefore, Researcher 1 had the flexibility to choose the datasets after looking at the preferred clusters in the proposed MIDR or MIDR (separate) system. Similarly, the results of the MIDR and MIDR (separate) systems could not be directly compared. Evaluation of the MIDR system was performed based on 10 datasets recommended for each researcher, whereas evaluation of MIDR (separate) system was performed based on 10 recommended datasets for each research field (cluster), which could be more than 10 datasets if a researcher had more than one research fields (clusters). Hence, the NDCG@10 and P@10 scores of MIDR (separate) system were less than the MIDR system. For researchers looking to find specific types of datasets, a keyword-based IR system might be more useful. For researcher who generally wanted to find datasets related to their interests, but did not have a particular interest in mind, could benefit from our system. For instance, if a researcher wanted a regular update of datasets relevant to their interest, our method would be better suited. However, this proposed system may not be useful to early-stage researchers due to fewer publications. They may take advantage of the available dataset retrieval systems such as DataMed, Omicseq and Google Dataset Search; or the text-based dataset searching that we provided on the website. Error analysisFor some clusters, evaluators rated all recommended datasets as one star. In most of these cases, we observed that the research field of that cluster was out of the scope of GEO. In this case, the NDCG@10 score was close to 1, but the P@10 score was 0. This may be one of the reasons why NDCG@10 scores were much higher compared to P@10 scores.  Screenshots of dataset recommendation system Researcher 1 (up) and Researcher 2 (down). Initially, we had not identified whether the clusters were related to GEO or not. We recommended datasets for these unrelated clusters. For example, Researcher 2 in Table 4 had a paper cluster which was related to statistical image analysis. For this specific cluster, Researcher 2 rated all the recommended datasets as one star, which reduced the scores of the systems. Later, we identified a threshold by averaging the similarity scores of publications and datasets for each cluster, and were able to remove the clusters which were not related to GEO. The threshold was set to 0.05 for the present study, i.e. a cluster was not considered for evaluation or showing recommendation if the average similarity score of the top 10 datasets for that cluster was less than or equals to 0.05. This threshold technique improved the results of proposed systems by 3% for Researcher 2. However, a thorough investigation on threshold involving datasets from different biomedical domains is needed for future work. Further, a dislike button for each cluster may be provided, and users may press the dislike button if that cluster is not related to GEO datasets. Later, this information can be used to build a machine learning-based system to identify and remove such clusters from further processing. This will improve the usefulness and reduce time complexity of the proposed recommendation system. . LimitationsThe researchers’ names are searched in PubMed to collect their publications. Many recent conference/journal publications are not updated in PubMed. Further, if the researcher has most of his/her publications that did not belong to the biomedical domain, then there is a low chance of getting those papers in PubMed. This makes the dataset recommendation task harder. Authors might later be able to include a subset of their non-PubMed articles for consideration in dataset recommendation (e.g. bioRxiv preprints), but this work is currently limited to PubMed publications only. We used PubMed name search to find the titles of a researcher’s papers. Finally, the titles were matched with the text in the CV to get publications. If there is any typo in the CV, then that publication would be rejected from being processed in further steps. As we do not fully parse the CV, instead just performing string matching to find publications, there is a high chance of rejecting publications with small typos. The manual evaluation was performed by five researchers only. For each cluster, 10 datasets were recommended, and each researcher has to evaluate an average of 40 datasets. It was a time-consuming task for evaluators to check each of the recommended datasets. For manual evaluation, we required the human judges with expertise on the GEO datasets, which was challenging to find. Further research will entail the scaling of this evaluation process. GETc PlatformWe developed the GETc research platform that recommends datasets to researchers using the proposed methods. A researcher needs to provide his/her name (as in PubMed) and CV (or list of publications) in the website. After processing his/her publications collected from PubMed, the recommendation system recommends datasets from GEO. Researchers can provide feedback for the datasets recommended by our system based on the evaluation criteria mentioned in Section 3.5 . A screenshot of the dataset recommendation system is shown in Figure 5 . This platform also recommends datasets using texts/documents, where cosine similarity of text and datasets are calculated, and datasets with a high score are recommended to users. Apart from dataset recommendation, it can also recommend literature and collaborators for each dataset. The platform analyzes time-course datasets using a specialized analysis pipeline (http://genestudy.org/pipeline) ( 25 ). We believe that these functions implemented in the GETc platform will significantly improve the reusability of datasets. This work is the first step toward developing a dataset recommendation tool to connect researchers to relevant datasets they may not otherwise be aware of. The maximum NDGC@10, P@10 (P) and P@10 (S) of 0.89, 0.78 and 0.61 were achieved based on the proposed method (MIDR) using five evaluators. This recommendation system will hopefully lead to greater biomedical data reuse and improved scientific productivity. Similar dataset recommendation can be developed for different datasets from both biomedical and other domains. The next goal is to identify the clusters which are not related to datasets and used for recommendations in the present article. These clusters can be removed from further experiments. Later, we plan to implement other embedding methods and test the dataset recommendation system on a vast number of users. A user-specific feedback-based system can be developed to remove datasets from the recommendations. Several additional dataset repositories can be added in the future. Other APIs can also be added to retrieve more complete representation of researcher’s publication history. Availability: http://genestudy.org/recommends/#/ We thank Drs. H.M, J.T.C, A.G and W.J.Z. for their help in evaluating the results and comments on designing that greatly improved the GETc research platform. This project is mainly supported by the Center for Big Data in Health Sciences (CBD-HS) at School of Public Health, University of Texas Health Science Center at Houston (UTHealth) and partially supported by the Cancer Research and Prevention Institute of Texas (CPRIT) project RP170 668 (K.R., H.W.) as well as the National Institute of Health (NIH) (grant R00LM012104) (K.R.). None declared. Chen X. et al. . ( 2018 ) Datamed–an open source discovery index for finding biomedical datasets . Journal of the American Medical Informatics Association , 25 , 300 – 308 . doi: 10.1093/jamia/ocx121 Google Scholar Roberts K. et al. . ( 2017 ) Information retrieval for biomedical datasets: the 2016 biocaddie dataset retrieval challenge . Database , 2017 , 1 – 9 . doi: 10.1093/database/bax068 Cohen T. et al. . ( 2017 ) A publicly available benchmark for biomedical dataset retrieval: the reference standard for the 2016 biocaddie dataset retrieval challenge . Database , 2017 , 1 – 10 . doi: 10.1093/database/bax061 Karisani P. , Qin Z.S. and Agichtein E. et al. ( 2018 ) Probabilistic and machine learning-based retrieval approaches for biomedical dataset retrieval . Database , 2018 , 1 – 12 . doi: 10.1093/database/bax104 Wright T.B. , Ball D. and Hersh W. et al. ( 2017 ) Query expansion using mesh terms for dataset retrieval: Ohsu at the biocaddie 2016 dataset retrieval challenge . Database , 2017 , 1 – 9 . doi: 10.1093/database/bax065 Scerri A. et al. . ( 2017 ) Elsevier’s approach to the biocaddie 2016 dataset retrieval challenge . Database , 2017 , 1 – 12 . doi: 10.1093/database/bax056 Wei W. et al. . ( 2018 ) Finding relevant biomedical datasets: the UC San Diego solution for the biocaddie retrieval challenge . Database , 2018 , 1 – 10 . doi: 10.1093/database/bay017 Sun X. et al. . ( 2017 ) Omicseq: a web-based search engine for exploring omics datasets . Nucleic acids research , 45 , W445 – W452 . doi: 10.1093/nar/gkx258 Jansen B.J. et al. . ( 2007 ) Determining the user intent of web search engine queries . In Proceedings of the 16th international conference on World Wide Web . ACM , Banff, Alberta, Canada pp. 1149 – 1150 . Achakulvisut T. et al. . ( 2016 ) Science concierge: A fast content-based recommendation system for scientific publications . PloS one , 11 , e0158423. doi: 10.1371/journal.pone.0158423 Patra B.G. et al. . ( 2020 ) A content-based literature recommendation system for datasets to improve data reusability. A case study on Gene Expression Omnibus (GEO) datasets . Journal of Biomedical Informatics , 104 , 103399. doi: 10.1016/j.jbi.2020.103399 Sansone S.-A. et al. . ( 2017 ) Dats, the data tag suite to enable discoverability of datasets . Scientific data , 4 , 170059. doi: 10.1038/sdata.2017.59 Ellefi M.B. et al. ( 2016 ) Dataset recommendation for data linking: An intensional approach . In European Semantic Web Conference . Springer , Heraklion, Crete, Greece pp. 36 – 51 . Nunes B.P. et al. ( 2013 ). Combining a co-occurrence-based and a semantic measure for entity linking . In Extended Semantic Web Conference , Springer , Montpellier, France 548 – 562 . Srivastava K.S. ( 2018 ). Predicting and recommending relevant datasets in complex environments . US Patent App. 15/721,122. Ghavimi B. et al. ( 2016 ) Identifying and improving dataset references in social sciences full texts . arXiv preprint arXiv:1603.01774 Positioning and Power in Academic Players, Agents and Agendas IOS Press 105 – 114 . doi: 10.3233/978-1-61499-649-1-105 Piwowar H.A. and Chapman W.W. ( 2008 ) Identifying data sharing in biomedical literature . In AMIA Annual Symposium Proceedings . American Medical Informatics Association , Washington, D.C., USA Vol. 2008 , p 596. Prasad A. et al. . ( 2019 ) Dataset mention extraction and classification . In Proceedings of the Workshop on Extracting Structured Knowledge from Scientific Publications . Association for Computational Linguistics , Minneapolis, Minnesota, USA pp 31 – 36 . Li Z. , Li J. and Yu P. et al. . ( 2018 ) Geometacuration: a web-based application for accurate manual curation of gene expression omnibus metadata . Database , 2018 , 1 – 8 . doi: 10.1093/database/bay019 Chen G. et al. ( 2019 ) Restructured geo: restructuring gene expression omnibus metadata for genome dynamics analysis . Database , 2019 , 1 – 8 . doi: 10.1093/database/bay145 Neal R.M. ( 2000 ) Markov chain sampling methods for dirichlet process mixture models . Journal of computational and graphical statistics , 9 , 249 – 265 . Yin J. and Wang J. ( 2016 ) A model-based approach for text clustering with outlier detection . In Proceedings of the 2016 IEEE 32nd International Conference on Data Engineering (ICDE) . IEEE , Helsinki, Finland , pp 625 – 636 . Lenoir T. and Giannella E. ( 2006 ) The emergence and diffusion of dna microarray technology . Journal of biomedical discovery and collaboration , 1 , 11. doi: 10.1186/1747-5333-1-11 Wang Y. et al. . ( 2013 ) A theoretical analysis of ndcg ranking measures . In 26th Annual Conference on Learning Theory (COLT 2013) PMLR Princeton, NJ, USA . Vol. 8 . Carey M. et al. . ( 2018 ) A big data pipeline: Identifying dynamic gene regulatory networks from time-course gene expression omnibus data with applications to influenza infection . Statistical methods in medical research , 27 , 1930 – 1955 . doi: 10.1177/0962280217746719 Author notesCitation details: Patra,B.G., Roberts,K., Wu,H., A content-based dataset recommendation system for researchers—a case study on Gene Expression Omnibus (GEO) repository. Database (2020) Vol. 00: article ID baaa064; doi:10.1093/database/baaa064
Email alertsCiting articles via.
Affiliations
Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide
This Feature Is Available To Subscribers OnlySign In or Create an Account This PDF is available to Subscribers Only For full access to this pdf, sign in to an existing account, or purchase an annual subscription. How To Make Recommendation in Case Study (With Examples) After analyzing your case study’s problem and suggesting possible courses of action , you’re now ready to conclude it on a high note. But first, you need to write your recommendation to address the problem. In this article, we will guide you on how to make a recommendation in a case study. Table of ContentsWhat is recommendation in case study, what is the purpose of recommendation in the case study, 1. review your case study’s problem, 2. assess your case study’s alternative courses of action, 3. pick your case study’s best alternative course of action, 4. explain in detail why you recommend your preferred course of action, examples of recommendations in case study, tips and warnings.  The Recommendation details your most preferred solution for your case study’s problem. After identifying and analyzing the problem, your next step is to suggest potential solutions. You did this in the Alternative Courses of Action (ACA) section. Once you’re done writing your ACAs, you need to pick which among these ACAs is the best. The chosen course of action will be the one you’re writing in the recommendation section. The Recommendation portion also provides a thorough justification for selecting your most preferred solution. Notice how a recommendation in a case study differs from a recommendation in a research paper . In the latter, the recommendation tells your reader some potential studies that can be performed in the future to support your findings or to explore factors that you’re unable to cover.  Your main goal in writing a case study is not only to understand the case at hand but also to think of a feasible solution. However, there are multiple ways to approach an issue. Since it’s impossible to implement all these solutions at once, you only need to pick the best one. The Recommendation portion tells the readers which among the potential solutions is best to implement given the constraints of an organization or business. This section allows you to introduce, defend, and explain this optimal solution. How To Write Recommendation in Case Study You cannot recommend a solution if you are unable to grasp your case study’s issue. Make sure that you’re aware of the problem as well as the viewpoint from which you want to analyze it .  Once you’ve fully grasped your case study’s problem, it’s time to suggest some feasible solutions to address it. A separate section of your manuscript called the Alternative Courses of Action (ACA) is dedicated to discussing these potential solutions. Afterward, you need to evaluate each ACA by identifying its respective advantages and disadvantages.  After evaluating each proposed ACA, pick the one you’ll recommend to address the problem. All alternatives have their pros and cons so you must use your discretion in picking the best among these ACAs. To help you decide which ACA to pick, here are some factors to consider:
You may also use a decision matrix to assist you in picking the best ACA 1 . This matrix allows you to rank the ACAs based on your criteria. Please refer to our examples in the next section for an example of a Recommendation formed using a decision matrix.  Provide your justifications for why you recommend your preferred solution. You can also explain why other alternatives are not chosen 2 .  To help you understand how to make recommendations in a case study, let’s take a look at some examples below. Case Study Problem : Lemongate Hotel is facing an overwhelming increase in the number of reservations due to a sudden implementation of a Local Government policy that boosts the city’s tourism. Although Lemongate Hotel has a sufficient area to accommodate the influx of tourists, the management is wary of the potential decline in the hotel’s quality of service while striving to meet the sudden increase in reservations. Alternative Courses of Action:
Recommendation: Upon analysis of the problem, it is recommended to implement ACA 1. Among all suggested ACAs, this option is the easiest to execute with the minimal cost required. It will not also impact potential profits and customers’ satisfaction with hotel service. Meanwhile, implementing ACA 2 might discourage customers from making reservations due to higher fees and look for other hotels as substitutes. It is also not recommended to do ACA 3 because reducing hotel services and privileges offered to customers might harm the hotel’s public reputation in the long run. The first paragraph of our sample recommendation specifies what ACA is best to implement and why. Meanwhile, the succeeding paragraphs explain that ACA 2 and ACA 3 are not optimal solutions due to some of their limitations and potential negative impacts on the organization. Example 2 (with Decision Matrix) Case Study: Last week, Pristine Footwear released its newest sneakers model for women – “Flightless.” However, the management noticed that “Flightless” had a mediocre sales performance in the previous week. For this reason, “Flightless” might be pulled out in the next few months. The management must decide on the fate of “Flightless” with Pristine Footwear’s financial performance in mind.
Decision Matrix Recommendation Based on the decision matrix above 3 , the best course of action that Pristine Wear, Inc. must employ is ACA 3 or selling “Flightless” shoes at lower prices to encourage more customers. This solution can be implemented immediately without the need for an excessive amount of financial resources. Since lower prices entice customers to purchase more, “Flightless” sales might perform better given a reduction in its price. In this example, the recommendation was formed with the help of a decision matrix. Each ACA was given a score of between 1 – 4 for each criterion. Note that the criterion used depends on the priorities of an organization, so there’s no standardized way to make this matrix. Meanwhile, the recommendation we’ve made here consists of only one paragraph. Although the matrix already revealed that ACA 3 tops the selection, we still provided a clear explanation of why it is the best.
Written by Jewel Kyle Fabula in Career and Education , Juander How  Jewel Kyle FabulaJewel Kyle Fabula is a Bachelor of Science in Economics student at the University of the Philippines Diliman. His passion for learning mathematics developed as he competed in some mathematics competitions during his Junior High School years. He loves cats, playing video games, and listening to music. Browse all articles written by Jewel Kyle Fabula Copyright NoticeAll materials contained on this site are protected by the Republic of the Philippines copyright law and may not be reproduced, distributed, transmitted, displayed, published, or broadcast without the prior written permission of filipiknow.net or in the case of third party materials, the owner of that content. You may not alter or remove any trademark, copyright, or other notice from copies of the content. Be warned that we have already reported and helped terminate several websites and YouTube channels for blatantly stealing our content. If you wish to use filipiknow.net content for commercial purposes, such as for content syndication, etc., please contact us at legal(at)filipiknow(dot)net  
Using Recommendation Systems in Disaster Management: A Systematic Literature ReviewNew citation alert added. This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. To manage your alert preferences, click on the button below. New Citation Alert!Please log in to your account Information & ContributorsBibliometrics & citations, view options, recommendations, design and evaluation of a role improvisation exercise for crisis and disaster response teams. This paper reports a case study, which aimed to investigate what role improvisation plays in crisis and disaster response teams and how improvisation performance of such teams can be simulated in representative contexts and constraints. The case study ... Dynamic and Time Critical Emergency Management for Level Three Disaster: A Case Study Analysis of Kerala Floods 2018Extreme variability of weather conditions has contributed to the increase in multi hazards events in Kerala, India. This work analyses the processes, systems and solutions used for management of a 24hr call center for an effective emergency response ... A Fuzzy Integer Programming Model to Locate Temporary Medical Facilities as Part of Pre-Disaster ManagementThe number and the scale of natural disasters have drastically increased over the last decades. One of the most vital stages of disaster preparedness is disaster response planning, and it plays an important role in limiting material and immaterial ... InformationPublished in. Elsevier Science Publishers B. V. Netherlands Publication HistoryAuthor tags.
ContributorsOther metrics, bibliometrics, article metrics.
View optionsLogin options. Check if you have access through your login credentials or your institution to get full access on this article. Full AccessShare this publication link. Copying failed. Share on social mediaAffiliations, export citations.
We are preparing your search results for download ... We will inform you here when the file is ready. Your file of search results citations is now ready. Your search export query has expired. Please try again. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
IMAGES
VIDEO
COMMENTS
The study of the recommendation system is a branch of information filtering systems (Recommender system, 2020). Information filtering systems deal with removing unnecessary information from the data stream before it reaches a human. Recommendation systems deal with recommending a product or assigning a rating to item.
Amazon is the largest e-commerce brand in the world in terms of revenue and market share. ( Statista) In 2021, Amazon's net revenue from e-commerce sales was US$470 billion, and about 35 percent of all sales on Amazon happen via recommendations. This clearly elucidates the power of recommendations. In this case study, we look at how Amazon is ...
On the practical side, integrating deep-learning toolboxes in our system has made it faster and easier to implement and experiment with both deep-learning and non-deep-learning approaches for various recommendation tasks. We conclude this article by summarizing our take-aways that may generalize to other applications beyond Netflix.
A Case Study on Recommendation Systems Based on Big Data: Proceedings of the Second International Conference on SCI 2018, Volume 2 January 2019 DOI: 10.1007/978-981-13-1927-3_44
Content-Based vs. Collaborative Filtering approaches for recommender systems. (Image by author) Content-Based Approach. Content-based methods describe users and items by their known metadata.Each item i is represented by a set of relevant tags—e.g. movies of the IMDb platform can be tagged as"action", "comedy", etc. Each user u is represented by a user profile, which can created from ...
As recommendation system algorithms, neural networks, and big data continue to evolve, their integration into various aspects of our lives is increasingly evident. ... S. Amer-Yahia, A comparative evaluation of top-n recommendation algorithms: Case study with total customers, in: 2020 IEEE International Conference on Big Data (Big Data), IEEE, ...
Abstract. This survey provides an exhaustive exploration of the evolution and current state of recommendation systems, which have seen widespread integration in various web applications. It focuses on the advancement of personalized recommendation strategies for online products or services. We categorize recommendation techniques into four ...
In this article, we outline some of the challenges encountered and lessons learned in using deep learning for recommender systems at Netflix. We first provide an overview of the various recommendation tasks on the Netflix service. We found that different model architectures excel at different tasks. Even though many deep-learning models can be ...
The goal of this chapter is to give an up-to-date overview of recommender systems techniques used in an industrial setting. We will give a high-level description the practical use of recommendation and personalization techniques. We will highlight some of the main lessons learned from the Netflix Prize. We will then use Netflix personalization ...
The six papers in this special issue push the current frontiers in recommender systems and address several of the challenges of open questions outlined above. In their article, Jannach and Chen ( 2022) elaborate why building a conversational recommender system is difficult, and consider such systems a "Grand AI Challenge".
We examined case studies conducted over the previous six years (2017-2022), with a focus on 35 key studies selected from 1938 academic papers found using the CADIMA tool. ... Baskota, A.; Ng, Y.K. A graduate school recommendation system using the multi-class support vector machine and KNN approaches. In Proceedings of the 2018 IEEE ...
7 Case Study—(Drug Recommendation System—Medicines) Due to the quick improvement of e-commerce, most of the user preferring to buy medicine online for the sake of comfort. Moveover user does not realize issue while purchasing drug without the knowledge and proper guideless. Major problem while purchasing medicines for online site namely ...
We observe a number of studies in recommendation systems that utilize a variety of approaches, such as rule-based recommendation systems, case-based reasoning-based recommendation systems, and hybrid recommendations systems using collaborative filtering-based algorithms [18,19,20,21,22,23,24]. However, in this particular study, we propose a ...
A detailed review of various recommendation systems is presented and typically recommender systems are based on the keyword search which allows the efficient scanning of very large document collections. The goal of a recommender system is to generate relevant recommendations for users. It is an information filtering technique that assists users by filtering the redundant and unwanted data from ...
Recommender system has the ability to predict whether a particular user would prefer an item or not based on the user's profile. Recommender systems are beneficial to both service providers and users [3]. They reduce transaction costs of finding and selecting items in an online shopping environment [4].
5 Use Case Scenarios for Recommendation Systems and How They Help. The market for recommendation engines is projected to grow from USD 1.14B in 2018 to 12.03B by 2025 with a CAGR of 32.39%, for the forecasted period. These figures are an indication of the growing emphasis on customer experience while also being a byproduct of the widespread ...
May 29, 2021. 1. In daily life we have used many social media application like FB, IG etc. but do you think how this platform will provide the friend recommendation alert. Also, the alert for the ...
The search keywords were as follows, using an inclusive OR: (recommender OR recommendation systems OR recommendation system) ... The authors also limited themselves to their specific case studies and did not make any recommendations for policy (last box plot is presented in Figure 2). All 73 studies reported the use of different data sets.
Recommendation system is playing a vital role in everyone's life as demand of recommendation for user's interest increasing day by day. ... 2021 Movie Recommendation Systems Based on Collaborative Filtering: A Case Study on Netflix Muhammed SÜTÇÜ 2 *1,2 , Ecem KAYA 2 , Oğuzkan ERDEM 2 *1 Abdullah Gül University, Faculty of Engineering ...
Read how a recommendation engine was built on the foundation of the Openkoda, utilizing a significant number of pre-existing features within the platform. ... Case studies: Recostream. Recommendation engine. august 2020. ... Once the system was validated and promoted from the experimental phase to production, the same team of developers was ...
Braja Gopal Patra, Kirk Roberts, Hulin Wu, A content-based dataset recommendation system for researchers—a case study on Gene Expression Omnibus (GEO) repository, Database, Volume 2020, 2020, ... The recommendation system can be viewed as an IR system where the most similar datasets can be retrieved for a researcher using his/her publications.
How To Write Recommendation in Case Study. 1. Review Your Case Study's Problem. 2. Assess Your Case Study's Alternative Courses of Action. 3. Pick Your Case Study's Best Alternative Course of Action. 4. Explain in Detail Why You Recommend Your Preferred Course of Action.
Selected studies were evaluated to provide an answer to five different research questions. We considered what are the most common recommendation approaches adopted in these studies, the input and output of the recommendation system, the disaster management phase involved, and the contribution of the study in the field.
21) did a survey study of government departments in Canada and the USA; the study showed PM was a system that is in constant need of improvement, and the study showed the balanced scorecard was a mainstream method for performance evaluation, numerous factors affect employee performance in organizations.