Scenario:

For my use case I have a set of financial documents.

The entire document set is divided into train,dev and test. The documents are parsed and the mentions and candidates are extracted with some rules.

The featurized training candidates are used to train a Fonduer Learning model and the model is used to predict on the test candidates, as per the normal fonduer pipeline as demonstrated in the hardware tutorial.

Problems & Questions

1. Is the fonduer prediction pipeline production ready? How can we fine tune it to achieve better accuracy? Should the main focus be on the quality of the extracted mentions?

With my initial analysis and usage following the hardware tutorial, I could not obtain good results.

2. Can we separate the training and test pipeline?

As in the current scenario, with a new document that I will feed for prediction, The entire corpus will have to be parsed to extract the mentions and candidates and store the feature keys.

Please correct me, if that won't be the case and help me with a snippet to showcase the separation.

Description of the problems or issues

Is your pull request related to a problem? Please describe. See #534. This request redoes #537, which needs prior fixing #538 (fixed by #539).

Does your pull request fix any issue. See #534

Description of the proposed changes

In case of multi line Japanese strings 'AAAA\nBBBB', spacy[ja] sometimes generates tokens ['AAA', 'AB', 'B', 'BB']. Proposal defines bbox of 'AB' as a multi line word (i.e. left is min left of ['A','B'], top is the top of 'A', right is max right of ['A','B'] and bottom is the bottom of 'B').

Test plan

This is cause of Japanese morphological analysis. So, I have added Japanese test data to 'tests/data/hocr_simple/japan.hocr' and test code to 'tests/parser/test_parser.py::test_parse_hocr'

Checklist

• [x] I have updated the documentation accordingly.
• [x] I have added tests to cover my changes.
• [x] All new and existing tests passed.
• [x] I have updated the CHANGELOG.rst accordingly.

Description of the bug

This is not a bug, but a performance issue. This is not noticeable when parsing a small number of documents, but parser.apply does not return even though the progress bar indicates it finishes parsing a long time ago (1 hour or more ago).

To Reproduce

Steps to reproduce the behavior:

1. Parse many documents (my case: ~2500)

Expected behavior

parser.apply returns when the progress bar indicates it finished parsing all the documents.

Error Logs/Screenshots

If applicable, add error logs or screenshots to help explain your problem.

Environment (please complete the following information)

• OS: Debian Buster
• PostgreSQL Version: 12.1
• Poppler Utils Version: N/A
• Fonduer Version: 0.8.3+dev (01e0d9319b523aff7aa7f5c583a9f330b0705ecc)

Additional context

Add any other context about the problem here.

Description of the problems or issues

Is your pull request related to a problem? Please describe.

Currently, preprocessor and parser are executed in a complete sequential order. i.e., preprocess N docs (and load them into a queue), then parse N docs. This has two drawbacks:

1. the progress bar shows nothing during preprocessing.
2. the machine RAM has to be large enough to hold N preprocessed docs at a time.

They become more serious when N is large and/or each HTML file is large.

Does your pull request fix any issue.

Fix #435

Description of the proposed changes

A clear and concise description of what you propose.

This PR

• places a cap on the in_queue so that only a certain number of documents are loaded to in_queue.
• executes preprocessor and parser in parallel (ie the main process does preprocessing and child process(es) do parsing in parallel).

Test plan

A clear and concise description of how you test the new changes.

For the 1st issue: I manually check the progress bar starts showing progress right after starting parse.apply.

Checklist

• [x] I have updated the documentation accordingly.
• [ ] I have added tests to cover my changes.
• [x] All new and existing tests passed.
• [x] I have updated the CHANGELOG.rst accordingly.

Hi,

In fonduer-tutorials, after running cell:

corpus_parser = OmniParser(structural=True, lingual=True, visual=True, pdf_path=pdf_path)
%time corpus_parser.apply(doc_preprocessor, parallelism=PARALLEL)

whenever is PARALLEL smaller than max_docs, I've got:

Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/multiprocessing/queues.py", line 240, in _feed
    send_bytes(obj)
  File "/anaconda3/lib/python3.6/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/anaconda3/lib/python3.6/multiprocessing/connection.py", line 398, in _send_bytes
    self._send(buf)
  File "/anaconda3/lib/python3.6/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

Otherwise (with PARALLEL bigger or equal than max_docs) result is empty tables in Postgresql. When turning off parallelisation, it works.

Best regards

Description of the problems or issues

The feature extraction only supports unary and binary candidates

Does your pull request fix any issue. Closes #455

Description of the proposed changes

Add new functions that supports multary-relation between spans for the feature extraction

Test plan

A clear and concise description of how you test the new changes. Use a candidate with more then two mentions, and try the feature extraction part.

Checklist

• [x] I have updated the documentation accordingly.
• [x] I have added tests to cover my changes.
• [x] All new and existing tests passed.
• [x] I have updated the CHANGELOG.rst accordingly.

Note:

In order for this to run the multary-candidates in textual features, we need a new version of treedlib based on this PR: treedlib#46 So if you can contact them, please do.

Also if someone can jump-in to improve the coverage, I can't get the tabular_features up

Description of the problems or issues

Is your pull request related to a problem? Please describe.

This is the second patch that follows #518 .

Does your pull request fix any issue.

N/A.

Description of the proposed changes

Add HOCRDocProprocessor and HocrVisualParser

Test plan

I added a few real hOCR example files.

Checklist

• [x] I have updated the documentation accordingly.
• [x] I have added tests to cover my changes.
• [x] All new and existing tests passed.
• [x] I have updated the CHANGELOG.rst accordingly.

Suppose that I have two mentions (say for example zip-code and tax code) whose matchers return true (checking 5 digit regex match for both mentions) for the same span in document, then I think Fonduer is throwing this error. please help me in resolving this.

sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "context_stable_id_key"
DETAIL:  Key (stable_id)=(1443208965_10_subset::span_mention:23313:23321) already exists.

[SQL: INSERT INTO context (type, stable_id) VALUES (%(type)s, %(stable_id)s) RETURNING context.id]

Hi

I am passing html to fonduer and it is saying unable read image from figure I have taken a pdf converted to html via pdftotree and passing the html to fonduer. Is this the issue with pdftotree that it is not able to render images. I want to what is the mechanism so that we can have images linked/embed in html so that fonduer can read it

Please help/advice as i am stuck with this issue

Describe the bug When working with large (~7k docs) corpus of hardware datasheets, extracting multiple relations, we expect that the features for each candidate would be deterministic between each run. Even more so if we have parallelism=1 set in the Featurizer. However, we find that there can be small (e.g., < 5) differences between feature tables, resulting in slightly different sparse matrices, and thus, slightly different results.

To Reproduce Running on the HACK transistor dataset will reproduce the error. However, it will take a long time, and we haven't been able to get a very minimal example that reproduces the error yet. Attached are two feature table dumps between two different runs with parallelism=1. Note that there is only a single difference on line 65454.

feature_table.tar.gz

Note that it isn't always one difference, and the difference is not deterministic. The different attached is just an example.

Expected behavior We would expect that these feature tables are identical between runs.

Error Logs/Screenshots For convenience, here is the differing line in screenshot form

Additional context If the issue is in the UDF implementation, this might affect the Labeler in addition to the Featurizer, since they share a lot of the UDF code.

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

I'm always frustrated when I have to look at the source codes to check the type of arguments/return.

Describe the solution you'd like A clear and concise description of what you want to happen.

1. Type hints (PEP484) are written to source codes like

def greeting(name: str) -> str:
    return 'Hello ' + name

2. (Eventually) enforce type checking during pre-commit

For example by flake8-mypy

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Depending on the editor (PyCharm, etc.), type/rtype documentation like below gives you type hinting. However, I'm not sure this is equivalent to the type hints (PEP484).

def greeting(name):
    """
    greeting

    :param name: description
    :type name: type description
    :return: description
    :rtype: type description
    """
    return 'Hello ' + name

Additional context Add any other context or screenshots about the feature request here.

Hello, thanks for providing this framework. My group has run into a bit of a snag:

For context, we've successfully completed candidate extraction & labeling for binary relations, with reasonable runtimes. With parallelism = 6, candidate extraction takes ~2 minutes per document.

We've since moved on to a 3-ary relation that is very similar to the binary relation. This 3-ary relation shares some mentions with the binary relation, and uses a very similar candidate extractor. We have done performance testing for the 3-ary throttler function and found it to have a very similar runtime to the binary throttler. Candidate extraction now takes 4 hours per document. This immense slowdown is due to the fact that computational complexity scales exponentially for each entity added to a relation.

Here are some numbers from our use case:

• Mention A: 800 mentions found
• Mention B: 140 mentions found
• Mention C: 150 mentions found

If our relation only includes (A,B), we have a total of 800*140 = 112,000 temporary candidates to evaluate with our throttler. Should we add mention C to form the relation (A,B,C), our total now grows to 800*140*150 = 16.8 million temporary candidates. We're unable to narrow our mention matchers further without excluding true positives.

This slowdown makes the Fonduer framework effectively unusable for any large-scale use case that requires relations with more than 2 entities. Can you provide guidance to address this issue?

Description of the bug

As part of iterative development in a Jupyter environment, apply may be re-run several times. The developer might need to update candidates or create a new labeling function, for example. When this happens, the corresponding Postgres table is cleared but not dropped. This means that the definition of the table cannot change to accommodate the updated parameters for apply.

To Reproduce

Steps to reproduce the behavior:

1. Run the max_storage_temp_tutorial notebook in fonduer-tutorials, up to and including the Labeling Functions section.
2. Add a new LF, doesn't need to do anything in particular (could return ABSTAIN every time). Add this to the stg_temp_lfs list.
3. Re-run the remainder of cells in the section.

Upon calling LFAnalysis, the following exception is thrown:

ValueError: Number of LFs (7) and number of LF matrix columns (6) are different

Expected behavior

Underlying tables for a re-run of a UDF apply method should not only be cleared, but dropped.

Error Logs/Screenshots

Full stack trace:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-62-e005feee6300> in <module>
      5 sorted_lfs = sorted(lfs, key=lambda lf: lf.name)
      6 
----> 7 LFAnalysis(L=L_train[0], lfs=sorted_lfs).lf_summary(Y=L_gold_train[0].reshape(-1))

~/.venv/lib/python3.7/site-packages/snorkel/labeling/analysis.py in __init__(self, L, lfs)
     44             if len(lfs) != self._L_sparse.shape[1]:
     45                 raise ValueError(
---> 46                     f"Number of LFs ({len(lfs)}) and number of "
     47                     f"LF matrix columns ({self._L_sparse.shape[1]}) are different"
     48                 )

ValueError: Number of LFs (7) and number of LF matrix columns (6) are different

Environment (please complete the following information)

• OS: Ubuntu 18.04
• PostgreSQL Version: 12.1
• Poppler Utils Version: 0.71.0-5
• Fonduer Version: 0.8.3

Additional context

https://github.com/HazyResearch/fonduer/issues/263#issuecomment-527588765 advises restarting Python, but this does not appear to solve the problem.

Description of the bug

I'm trying to Train a model that can build a Knowledge Base from the OPC UA Companions specification as a part of my Thesis. I have the Dataset as PDFs and used a third-party program to convert them into HTML and tried my best to preserve the data structure information (i'm getting the same result even if i just Parsed on the PDFs alone).

Then i followed the hardware_fonduer_model Tutorial to Extract the Candidates accordingly.

the Problem is that the Parser is splitting the sentences wrongly, namely it is getting the end of a Line as an end of a sentence. I tried to debug using a SimpleParser.split_sentences(text) command and turned out that python needs a backslash to split a statement into multiple lines.

So i thought maybe i could use the replacements=['[\n]', ' '] parameter so the Split could function better but i'm getting the ValueError: too many values to unpack (expected 2). What is the default configuration for the sentence segmentation?
How could i get a multiple Sentences as a mention? (i tried MentionNgram till n_max =100 and still getting just one).

I would really appreciate getting back from you.

many thanks in advance

Example: Text to be parsed

Boolean indicating if a profile /signature should be generated by this move command request.If the optional VariableSignatureRequestStatus is not provided on the Object, this parameter is ignored by the Server.

Expected behavior

sentence 1 : Boolean indicating if a profile /signature should be generated by this move command request. sentence 2 : If the optional VariableSignatureRequestStatus is not provided on the Object, this parameter is ignored by the Server.

Actual behavior

sentence 1 : Boolean indicating if a profile /signature should be generated by this move command sentence 2 : request. sentence 3 : request.If the optional VariableSignatureRequestStatus is not provided on the Object, this sentence 4 : parameter is ignored by the Server.

Environment

• OS: Ubuntu 20.04.1 LTS
• PostgreSQL Version: 12.0
• Poppler Utils Version: 0.2.1]
• Fonduer Version: 0.8.2 MDISCompanionSpecification.pdf

@SenWu @HiromuHota .. can you pls suggest if my analogy is right?

I am getting error :- File "abcd./anaconda3/lib/python3.7/site-packages/fonduer/utils/data_model_utils/structural.py", line 55, in _get_node return doc_etree.xpath(sentence.xpath)[0] IndexError: list index out of range

I am following Hardware tutorial on some Email HTML msgs and getting mentions count near 4000

Also :-- train_cands = candidate_extractor.get_candidates(split=0) dev_cands = candidate_extractor.get_candidates(split=1) test_cands = candidate_extractor.get_candidates(split=2)

Above steps returned outputs but,

on applying featurizer: featurizer.apply(split=0, train=True, parallelism=PARALLEL)

I am getting error mentioned on top.

I looked stackoverflow but the reason that HTML syntax issue,.. is not there as it is rendering good on browser. So can you share your thoughts on :

1. can it be because no candidates being generated? or
2. something else

Thanks.

How can i extract a paragraph and all associated sentences in document
Basically i need paragraphs with associated sentences @lukehsiao @SenWu @vincentschen @ZZWENG @stephenbach

Appreciate your help

Description of the bug

Unlike other methods (eg Featurizer.drop_keys() and Featurizer.upsert_keys(), Featurizer.get_keys() does not honor candidate classes in context but returns all feature keys no matter which candidate class they are associated with. This is confusing.

See https://github.com/HazyResearch/fonduer/issues/511#issuecomment-696618392 for how this actually confused a user.

To Reproduce

This is a design error.

Expected behavior

These methods should behave similarly. Either

• None of these honor candidate classes, or
• All of these honor them.

Error Logs/Screenshots

N/A

Environment (please complete the following information)

• Fonduer Version: 0.8.3

Additional context

Add any other context about the problem here.