Easily Add Recommender system to your project

Gorse is an open-source recommender system service written in Go. You can add this easily to your project because have Restful Apis and Dotnet SDK and etc.

Home

In this article I want to talk about Dotnet SDK and use this in dotnet applications.

Gorse Panel

Features

Multi-source : Recommend items from Popular, latest, user-based, item-based and collaborative filtering.

AutoML : Search the best recommendation model automatically in the background.

Distributed prediction : Support horizontal scaling in the recommendation stage after single node training.

RESTful APIs : Expose RESTful APIs for data CRUD and recommendation requests.

Multi-database support : Support Redis, MySQL, Postgres, MongoDB, and ClickHouse as its storage backend.

Online evaluation : Analyze online recommendation performance from recently inserted feedback.

Dashboard : Provide GUI for data management, system monitoring, and cluster status checking.

Open source : The codebase is released under Apache 2 license and driven by the community.

Gorse is an open-source recommendation system written in Go. Gorse aims to be a universal open-source recommender system that can be easily introduced into a wide variety of online services. By importing items, users and interaction data into Gorse, the system will automatically train models to generate recommendations for each user.

Docker

This guide walks you through the quickest way to setup a recommender system for GitHub repositories based on dataset from GitRecopen in new window. Make sure you have installed the following softwares at the beginning:

• Dockeropen in new window, nodes will run in containers.
• Docker Composeopen in new window, nodes will be orchestrated by Docker Compose.

Dotnet Gorse Client Sample

using Gorse.NET;

var client = new Gorse("http://127.0.0.1:8087", "api_key");

client.InsertFeedback(new Feedback[]
{
    new Feedback{FeedbackType="star", UserId="bob", ItemId="vuejs:vue", Timestamp="2022-02-24"},
    new Feedback{FeedbackType="star", UserId="bob", ItemId="d3:d3", Timestamp="2022-02-25"},
    new Feedback{FeedbackType="star", UserId="bob", ItemId="dogfalo:materialize", Timestamp="2022-02-26"},
    new Feedback{FeedbackType="star", UserId="bob", ItemId="mozilla:pdf.js", Timestamp="2022-02-27"},
    new Feedback{FeedbackType="star", UserId="bob", ItemId="moment:moment", Timestamp="2022-02-28"},
});

Then, fetch 10 recommended items from Gorse. We can find that frontend-related repositories are recommended for Bob.

client.GetRecommend("10");

Data Objects

There are only three kinds of data objects in Gorse: users, items and feedbacks.

User

A user object consists of a user ID and labels describing the user. The user labels can be empty, but these labels help to improve recommendations.

type User struct {
    UserId    string // user ID
    Labels    []string // labels describing the user
}

Item

An item consists of 6 fields:

type Item struct {
 ItemId     string
 IsHidden   bool
 Categories []string
 Timestamp  time.Time
 Labels     []string
 Comment    string
}

Gorse Data Objects

Multi-Categories Recommendation

Multi-categories recommendations are common, take YouTube for example, where multiple recommendation categories are provided on the homepage.

Multi Categories

Multiple categories can be distinguished by topics such as food, travel, etc., or by forms, e.g. live, short and long videos. Items will appear in the global recommendation stream, and in addition, the `Categories`` field determines which recommendation categories the items should appear in. For each recommendation API, there are a global version and a categorized version:

End Points

Feedback

A feedback consists of user ID, item ID, feedback type, and feedback timestamp, where the triad of user ID, item ID, and feedback type is required to be unique in the database.

type Feedback struct {
    FeedbackType string  // feedback type
    UserId       string  // user id
    ItemId       string  // item id
    Timestamp    time.Time // feedback timestamp
}

.NET SDK

• Install via .NET CLI:

dotnet add package Gorse.NET

Usage

using Gorse.NET;

var client = new Gorse("http://127.0.0.1:8087", "api_key");

client.InsertFeedback(new Feedback[]
{
    new Feedback{FeedbackType="star", UserId="bob", ItemId="vuejs:vue", Timestamp="2022-02-24"},
    new Feedback{FeedbackType="star", UserId="bob", ItemId="d3:d3", Timestamp="2022-02-25"},
    new Feedback{FeedbackType="star", UserId="bob", ItemId="dogfalo:materialize", Timestamp="2022-02-26"},
    new Feedback{FeedbackType="star", UserId="bob", ItemId="mozilla:pdf.js", Timestamp="2022-02-27"},
    new Feedback{FeedbackType="star", UserId="bob", ItemId="moment:moment", Timestamp="2022-02-28"},
});

client.GetRecommend("10");

RestFull APIs

RESTful APIs

Architecture

The master node loads data from the database. In the process of loading data, popular items and the latest items are written to the cache. Then, the master node searches for neighbors and training recommendation models. In the background, the random search is used to find the optimal recommendation model for current data. The worker nodes pull recommendation models from the master node and generate recommendations for each user. The server nodes provide RESTful APIs. Workers and servers connect to the master node via GRPC, which is configured in the configuration file.

[master]

# GRPC port of the master node. The default value is 8086.
port = 8086

# gRPC host of the master node. The default values is "0.0.0.0".
host = "0.0.0.0"

# Number of working jobs in the master node. The default value is 1.
n_jobs = 1

# Meta information timeout. The default value is 10s.
meta_timeout = "10s"

The number of working jobs for the master node is set by master.n_jobs. For workers, the number of working jobs is set by the command line flag.

Recommendation Flow

Gorse works like a waterfall. Users, items and feedbacks are the sources of water. Intermediate results will be cached in cache storage (eg. Redis) such as water falling from the first waterfall will be cached in the intermediate pool. After several stages, selected items are recommended to users.

The intermediate cache is configurable. Increasing cache size might improve recommendations since the recommender system has more information on data but also consumes more cache storage. The expiration time of the cache should be traded off between freshness and CPU costs.

[recommend]

# The cache size for recommended/popular/latest items. The default value is 10.
cache_size = 100

# Recommended cache expire time. The default value is 72h.
cache_expire = "72h"