An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Last update: Dec 02, 2022

Related tags

Deep Learning recsys_metrics

Overview

recsys_metrics

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Overview • Installation • How to use • Benchmark • Citation •

Overview

Highlights

Efficient (vectorized) implementations over mini-batches
Standard RecSys metrics: precision, recall, map, mrr, hr, ndcg
Beyond-accuracy metrics: e.g. coverage, diversity, novelty, etc.
All metrics support a top-k argument.

Why do we need `recsys_metrics`?

Installation

You can install recsys_metrics from PyPI:

pip install recsys_metrics

Or you can also install the latest version from source:

pip install git+https://github.com/zuoxingdong/[email protected]

Note that we support Python 3.7+ only.

How to use

Let us take Hit Rate (HR) to illustrate how to use this library:

preds = torch.tensor([
    [.5, .3, .1],
    [.3, .4, .5]
])
target = torch.tensor([
    [0, 0, 1],
    [0, 1, 1]
])
hit_rate(preds, target, k=1, reduction='mean')

>> tensor(0.5000)

The first example in the batch does not have a hit (i.e. top-1 item is not a relevant item) and second example in the batch gets a hit (i.e. top-1 item is a relevant item). Thus, we have a hit-rate of 0.5.

The API of other metrics are of the same format.

Benchmark

Metrics	Single Example	Mini-Batch
Precision
Recall
MAP
MRR
HR
NDCG

Citation

This work is inspired by Torchmetrics from PyTorchLightning Team.

Please use this bibtex if you want to cite this repository in your publications:

@misc{recsys_metrics,
      author = {Zuo, Xingdong},
      title = {recsys_metrics: An efficient PyTorch implementation of the evaluation metrics in recommender systems.},
      year = {2021},
      publisher = {GitHub},
      journal = {GitHub repository},
      howpublished = {\url{https://github.com/zuoxingdong/recsys_metrics}},
    }

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

NVIDIA Merlin NVIDIA Merlin is an open source library designed to accelerate recommender systems on NVIDIA’s GPUs. It enables data scientists, machine

419 Jan 3, 2023

This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard e

71 Dec 30, 2022

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking Datasets You can download datasets that have been pre-pr

25 May 29, 2022

Active and Sample-Efficient Model Evaluation

Active Testing: Sample-Efficient Model Evaluation Hi, good to see you here! 👋 This is code for "Active Testing: Sample-Efficient Model Evaluation". P

19 Oct 30, 2022

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

引言感谢苏神带来的模型，原文地址：https://spaces.ac.cn/archives/8877 如何运行对应模型EfficientGlobalPoi

40 Dec 14, 2022

UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

[ICLR 2021] "UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems" by Jiayi Shen, Haotao Wang*, Shupeng Gui*, Jianchao Tan, Zhangyang Wang, and Ji Liu

39 Dec 3, 2022

Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering recommendation algorithms in the world of scientiﬁc Python packages (numpy, scipy, matplotlib).

Crab - A Recommendation Engine library for Python Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering r

1.2k Dec 21, 2022

A python library for implementing a recommender system

python-recsys A python library for implementing a recommender system. Installation Dependencies python-recsys is build on top of Divisi2, with csc-pys

1.5k Dec 17, 2022

A TikTok-like recommender system for GitHub repositories based on Gorse

GitRec GitRec is the missing recommender system for GitHub repositories based on Gorse. Architecture The trending crawler crawls trending repositories

337 Jan 4, 2023

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Related tags

Overview

Overview

Highlights

Why do we need recsys_metrics?

Installation

How to use

Benchmark

Citation

You might also like...

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

Active and Sample-Efficient Model Evaluation

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering recommendation algorithms in the world of scientiﬁc Python packages (numpy, scipy, matplotlib).

A python library for implementing a recommender system

A TikTok-like recommender system for GitHub repositories based on Gorse

Releases(v0.0.3)

v0.0.3(Aug 1, 2022)

v0.0.2(Jun 14, 2022)

Owner

Xingdong Zuo

UIUCTF 2021 Public Challenge Repository

Python package for missing-data imputation with deep learning

Repository of Jupyter notebook tutorials for teaching the Deep Learning Course at the University of Amsterdam (MSc AI), Fall 2020

MMdet2-based reposity about lightweight detection model: Nanodet, PicoDet.

Malware Env for OpenAI Gym

Video Frame Interpolation without Temporal Priors (a general method for blurry video interpolation)

Video2x - A lossless video/GIF/image upscaler achieved with waifu2x, Anime4K, SRMD and RealSR.

Research on controller area network Intrusion Detection Systems

Pytorch implementation of Hinton's Dynamic Routing Between Capsules

Computer Vision application in the web

Using Language Model to Bootstrap Human Activity Recognition Ambient Sensors Based in Smart Homes

DetCo: Unsupervised Contrastive Learning for Object Detection

Implementation of Uformer, Attention-based Unet, in Pytorch

Official Implementation of Few-shot Visual Relationship Co-localization

Code for the published paper : Learning to recognize rare traffic sign

Unofficial implementation of MLP-Mixer: An all-MLP Architecture for Vision

Implementation for "Seamless Manga Inpainting with Semantics Awareness" (SIGGRAPH 2021 issue)

Training neural models with structured signals.

Random Erasing Data Augmentation. Experiments on CIFAR10, CIFAR100 and Fashion-MNIST

Deep Image Matting implementation in PyTorch

Why do we need `recsys_metrics`?