This is a continuation of https://github.com/explosion/spaCy/pull/7264, since we decided to add the coref components here first. It's still a work in progress.

This PR adds three new experimental suggester functions for the spancat component and a spaCy project showcasing how to use them in a config.cfg file.

Subtree Suggester:

• Uses annotations from the Tagger and Parser to suggests subtrees of individual tokens

Chunk Suggester:

• Uses annotations from the Tagger and Parser to suggest noun_chunks

Sentence Suggester:

• Uses sentence boundaries to suggest sentences

These suggesters also come with the ngram functionality which allows users to set a list of sizes for suggesting individual ngrams

The spaCy project covers:

• How to source components from existing models
• How to use frozen_components & annotating_components
• How to use custom suggester functions registered in the registry

This PR adds a new experimental component for learning span boundaries and a custom suggester function for spancat. It further adds a spaCy project showcasing how to use the SpanFinder component on 3 different datasets (Healthsea, ToxicSpans, Genia) with 2 configurations (tok2vec & transformer). The project also provides the possibility to train spancat with ngram and compare it to SpanFinder with a custom evaluation script that calculates the performance and overall coverage of the suggester functions.

Features

• spaCy project for comparing SpanFinder vs Ngram
• SpanFinder model
• SpanFinder component
• SpanFinder suggester
• Unit tests for component, model and suggester

Docs with one or zero tokens fail in the coref component. This doesn't have a fix yet, just a failing test. (There is also a test for the span resolver, which does not fail.)

The SpanResolver seems unable to include the final token in a Doc in output spans. It will even produce empty spans instead of doing so.

This makes changes so that within the model span end indices are treated as inclusive, and converts them back to exclusive when annotating docs. This has been tested to work, though an automated test should be added.

Before this PR, in environments without PyTorch, using spacy experimental can fail due to attempts to load entry points. This change makes it so the types required for class definitions (torch.nn.Module and torch.Tensor) are stubbed to object when torch is not available.

It looks like Torch 1.13.0 has some changes in the way devices are handled and can result in subtle errors in code that worked previously. This explicitly specifies a device in one place, and may resolve https://github.com/explosion/spaCy/issues/11734. For another example of this issue, see https://github.com/pytorch/pytorch/issues/85450.

The core problem is that a CPU tensor is being indexed using a non-CPU tensor.

Leaving as a draft in case this doesn't resolve this issue, which might happen if we're doing the same thing somewhere else.

spacy-experimental is supposed to be safe to load without PyTorch, even if large parts of it aren't functional, but that wasn't checked in tests. This adds a check for that by simply running the tests before installing PyTorch and again afterwards.

Given the current state of master, which doesn't have #23, this should fail.

Coerce scalar tensors to native Python integers to avoid comparison overhead.

With the above change (and another optimization to SpanGroups.copy; PR), we see a 90% reduction in execution time of the set_annotation pipeline phase.

Before

After

Copy the entire mentions array to the CPU, create tuple keys on-demand, pre-allocate output matrix as Float2d. Also remove unnecessary casts in CoreferenceResolver.update.

Before

After

Embedding component that is the deterministic version of MultiHashEmbed i.e.: each token gets mapped to an index unless they are not in the vocabulary in which case they get mapped to a learned unknown vector.

The mechanism to initialize MultiEmbed is a bit strange. The Model gets created first with dummy Embed layers. Then when init gets called MultiEmbed expects the model.attrs["tables"] to be already set, which provides the mapping from token attributes to indices. During initialization the dummy Embed layers get replaced by ones that adjust their sizes to the number of symbols in the tables.

A helper callback is provided in set_attr.py that should be placed in the initialize.before_init section in the config. It can be used to set the tables for MultiEmbed.

Currently the token_map.py is a script that has the structure of the usual spacy init scrips.

We use sentence splitting in the biaffine parser to keep the O(n^2) biaffine attention model tractable. However, since the sentence splitter makes errors, the parser may not have the correct head available.

This change adds another splitting strategy as the preferred splitting. The goal of this strategy is to split up a Doc into pieces that are as large as possible given a maximum n_max. This reduces the number of attachment errors as a result of incorrect sentence splits, while providing an upper bound on complexity (O(n_max^2)).

The algorithm works as follows:

• If the length |d| > max_length:
  • Find the highest-probability split in d according to senter.
  • Split d into d_1 and d_2 using the highest probability split.
  • Recursively apply this algorithm to d_1 and d_2.
• Otherwise: do nothing

Note: draft, requires functionality from PR https://github.com/explosion/spaCy/pull/11002, which targets spaCy v4.