LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

A Python package for causal inference using Synthetic Controls

CycleTransGAN-EVC: A CycleGAN-based Emotional Voice Conversion Model with Transformer

PyTorch Connectomics: segmentation toolbox for EM connectomics

This repository is the code of the paper Accelerating Deep Reinforcement Learning for Digital Twin Network Optimization with Evolutionary Strategies

Bayesian Optimization using GPflow

Task-based end-to-end model learning in stochastic optimization

Repository of our paper 'Refer-it-in-RGBD' in CVPR 2021

darija <-> english dictionary

Simple Dynamic Batching Inference

Objax Apache-2Objax (🥉19 · ⭐ 580) - Objax is a machine learning framework that provides an Object.. Apache-2 jax

AOT-GAN for High-Resolution Image Inpainting (codebase for image inpainting)

Code for the paper "Generative design of breakwaters usign deep convolutional neural network as a surrogate model"

Pytorch implementation of paper: "NeurMiPs: Neural Mixture of Planar Experts for View Synthesis"

CN24 is a complete semantic segmentation framework using fully convolutional networks

TensorFlow implementation of ENet

Avalanche RL: an End-to-End Library for Continual Reinforcement Learning

DeLighT: Very Deep and Light-Weight Transformers

Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data

Mosaic of Object-centric Images as Scene-centric Images (MosaicOS) for long-tailed object detection and instance segmentation.

Differentiable Factor Graph Optimization for Learning Smoothers @ IROS 2021