Utils to quickly evaluate many 🤗 models on the GLUE tasks

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/
Owner
Przemyslaw K. Joniak
Grad student at The University of Tokyo
Przemyslaw K. Joniak
Minecraft Multi-Server Pinger Discord Embed

Minecraft Network Pinger Minecraft Multi-Server Pinger Discord Embed What does this bot do? It sends an embed and uses mcsrvstat API and checks if the

YungHub 2 Jan 05, 2022
pyreports is a python library that allows you to create complex report from various sources

pyreports pyreports is a python library that allows you to create complex reports from various sources such as databases, text files, ldap, etc. and p

Matteo Guadrini aka GU 78 Dec 13, 2022
Moleey Panel with python 3

Painel-Moleey pkg upgrade && pkg update pkg install python3 pip install pyfiglet pip install colored pip install requests pip install phonenumbers pkg

Moleey. 1 Oct 17, 2021
This repo is a collection of programs and websites templates too

📢 Register here for Hacktoberfest and make four pull requests (PRs) between October 1st-31st to grab free SWAGS 🔥 . IMPORTANT While making pull requ

Binayak Jha - 2 7 Oct 03, 2022
a url shortener with fastapi and tortoise-orm

fastapi-tortoise-orm-url-shortener a url shortener with fastapi and tortoise-orm

19 Aug 12, 2022
A dead-simple service that notifies you when something goes down.

Totmannschalter Totmannschalter (German for dead man's switch) is a simple service that notifies you when it has not received any message from a servi

1 Dec 20, 2021
take home quiz

guess the correlation data inspection a pretty normal distribution train/val/test split splitting amount .dataset: 150000 instances ├─8

HR Wu 1 Nov 04, 2021
Simple python bot, that notifies about new manga chapters through Telegram.

Simple python bot, that notifies about new manga chapters through Telegram.

Dmitry Kopturov 1 Dec 05, 2021
Python library for parsing Godot scene files

Godot Parser This is a python library for parsing Godot scene (.tscn) and resource (.tres) files. It's intended to make it easier to automate certain

Steven Arcangeli 30 Jan 04, 2023
Abilian Core: an enterprise application development platform based on the Flask micro-framework, the SQLAlchemy ORM

About Abilian Core is an enterprise application development platform based on the Flask micro-framework, the SQLAlchemy ORM, good intentions and best

Abilian open source projects 47 Apr 14, 2022
Python program that generates random user from API

RandomUserPy Author kirito sate #modules used requests, json, tkinter, PIL, urllib, io, install requests and PIL modules from pypi pip install Pillow

kiritosate 1 Jan 05, 2022
Combines power of torch, numerical methods to conquer and solve ALL {O,P}DEs

torch_DE_solver Combines power of torch, numerical methods and math overall to conquer and solve ALL {O,P}DEs There are three examples to provide a li

Natural Systems Simulation Lab 28 Dec 12, 2022
Compiler Final Project - Lisp Interpreter

Compiler Final Project - Lisp Interpreter

2 Jan 23, 2022
NeurIPS'19: Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (Pytorch implementation for noisy labels).

Meta-Weight-Net NeurIPS'19: Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (Official Pytorch implementation for noisy labels). The

243 Jan 03, 2023
Tiling manager which runs on top of EWMH window managers.

PyTyle is an extremely versatile and extensible tiling manager that is meant to be used on top of EWMH window managers. Its feature set was modeled af

55 Jul 29, 2021
Gobigger Explore For Python

Gobigger-Explore 🔮 GoBigger Challenge 2021 Baseline en/中文 🤖 Introduction This is the baseline of GoBigger Multi-Agent Decision Intelligence Challeng

OpenDILab 145 Dec 22, 2022
In this repo, I will put all the code related to data science using python libraries like Numpy, Pandas, Matplotlib, Seaborn and many more.

Python-for-DS In this repo, I will put all the code related to data science using python libraries like Numpy, Pandas, Matplotlib, Seaborn and many mo

1 Jan 10, 2022
This code extracts line width of phonons from specular energy density (SED) calculated with LAMMPS.

This code extracts line width of phonons from specular energy density (SED) calculated with LAMMPS.

Masato Ohnishi 3 Jun 15, 2022
pydock - Docker-based environment manager for Python

pydock - Docker-based environment manager for Python ⚠️ pydock is still in beta mode, and very unstable. It is not recommended for anything serious. p

Alejandro Piad 16 Sep 18, 2021
A tutorial presents several practical examples of how to build DAGs in Apache Airflow

Apache Airflow - Python Brasil 2021 Este tutorial apresenta vários exemplos práticos de como construir DAGs no Apache Airflow. Background Apache Airfl

Jusbrasil 14 Jun 03, 2022