Code for Massive-scale Decoding for Text Generation using Lattices

Last update: Dec 18, 2022

Related tags

Overview

Massive-scale Decoding for Text Generation using Lattices

TL;DR: a new search algorithm to construct lattices encoding many generation options; two key technical contributions: (1) best-first search, (2) path recombination.

Visualization

We provide a few examples in the vis folder and on my homepage. You need to download the html files to view and interact with the model outputs.

The complete set of outputs are available on Box.

Getting started

model contains all of the methods, including baselines like beam search, nucleus sampling, and our methods.
evaluation contains scripts for evaluation.
command are the prompts and shells we use to run the experiment.

Beam Search:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4  -beam_size 16 -min_len 10 -max_len 35   -model bs

Best-first Search:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4  -beam_size 16 -min_len 10 -max_len 35   -model astar_baseline

Best-first Search with Recomb:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4 -beam_size 16 -min_len 10 -max_len 35 -model astar -merge imp  -avg_score 0.75  -adhoc

Best-first Search with Zip:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4 -beam_size 16 -min_len 10 -max_len 35 -model astar -merge zip  -avg_score 0.75  -adhoc

More detailed instructions coming soon!

Citation

@misc{xu-durrett-2021-massive,
    title={Massive-scale Decoding for Text Generation using Lattices},
    author={Jiacheng Xu and Greg Durrett},
    year={2021},
    eprint={2112.07660},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Contact

[email protected]

Code for Massive-scale Decoding for Text Generation using Lattices

Related tags

Overview

Massive-scale Decoding for Text Generation using Lattices

Visualization

Getting started

Citation

Contact

Owner

Jiacheng Xu

[NeurIPS 2021] Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods

PyTorch implementation for View-Guided Point Cloud Completion

Gradient Step Denoiser for convergent Plug-and-Play

Evaluation suite for large-scale language models.

Python implementation of Lightning-rod Agent, the Stack4Things board-side probe

🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

Implementation of the state-of-the-art vision transformers with tensorflow

Face detection using deep learning.

Self-supervised Deep LiDAR Odometry for Robotic Applications

A solution to ensure Crowd Management with Contactless and Safe systems.

Learning where to learn - Gradient sparsity in meta and continual learning

ArcaneGAN by Alex Spirin

Huawei Hackathon 2021 - Sweden (Stockholm)

This is a library for training and applying sparse fine-tunings with torch and transformers.

LabelImg is a graphical image annotation tool.

[NeurIPS 2020] This project provides a strong single-stage baseline for Long-Tailed Classification, Detection, and Instance Segmentation (LVIS).

Optimizing DR with hard negatives and achieving SOTA first-stage retrieval performance on TREC DL Track (SIGIR 2021 Full Paper).

Official implementation of the paper ``Unifying Nonlocal Blocks for Neural Networks'' (ICCV'21)

Train emoji embeddings based on emoji descriptions.

[ACM MM 2021] Multiview Detection with Shadow Transformer (and View-Coherent Data Augmentation)