txtai executes machine-learning workflows to transform data and build AI-powered semantic search applications.

Hello community, it would be very nice to make a .Net binding. Where can I find a guide for that? So please if anyone can give me some instructions on how to get in that would be great. Of course, I would also like to know if this binding issue is only handled by maintainers or if it is something open to the community.

Hello, when I try to run the indexing step, I get this error.

Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-33-6e863ca8aecc> in <module>
----> 1 embeddings.index(to_index)

~\Anaconda3\envs\bert2\lib\site-packages\txtai\embeddings.py in index(self, documents)
     80 
     81         # Transform documents to embeddings vectors
---> 82         ids, dimensions, stream = self.model.index(documents)
     83 
     84         # Load streamed embeddings back to memory

~\Anaconda3\envs\bert2\lib\site-packages\txtai\vectors.py in index(self, documents)
    245                 if len(batch) == 500:
    246                     # Convert batch to embeddings
--> 247                     uids, dimensions = self.batch(batch, output)
    248                     ids.extend(uids)
    249 

~\Anaconda3\envs\bert2\lib\site-packages\txtai\vectors.py in batch(self, documents, output)
    279 
    280         # Build embeddings
--> 281         embeddings = self.model.encode(documents, show_progress_bar=False)
    282         for embedding in embeddings:
    283             if not dimensions:

~\Anaconda3\envs\bert2\lib\site-packages\sentence_transformers\SentenceTransformer.py in encode(self, sentences, batch_size, show_progress_bar, output_value, convert_to_numpy, convert_to_tensor, device, normalize_embeddings)
    192 
    193             with torch.no_grad():
--> 194                 out_features = self.forward(features)
    195 
    196                 if output_value == 'token_embeddings':

~\Anaconda3\envs\bert2\lib\site-packages\torch\nn\modules\container.py in forward(self, input)
    117     def forward(self, input):
    118         for module in self:
--> 119             input = module(input)
    120         return input
    121 

~\Anaconda3\envs\bert2\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

~\Anaconda3\envs\bert2\lib\site-packages\sentence_transformers\models\Transformer.py in forward(self, features)
     36             trans_features['token_type_ids'] = features['token_type_ids']
     37 
---> 38         output_states = self.auto_model(**trans_features, return_dict=False)
     39         output_tokens = output_states[0]
     40 

~\Anaconda3\envs\bert2\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

~\Anaconda3\envs\bert2\lib\site-packages\transformers\models\bert\modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)
    962         head_mask = self.get_head_mask(head_mask, self.config.num_hidden_layers)
    963 
--> 964         embedding_output = self.embeddings(
    965             input_ids=input_ids,
    966             position_ids=position_ids,

~\Anaconda3\envs\bert2\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

~\Anaconda3\envs\bert2\lib\site-packages\transformers\models\bert\modeling_bert.py in forward(self, input_ids, token_type_ids, position_ids, inputs_embeds, past_key_values_length)
    205         if self.position_embedding_type == "absolute":
    206             position_embeddings = self.position_embeddings(position_ids)
--> 207             embeddings += position_embeddings
    208         embeddings = self.LayerNorm(embeddings)
    209         embeddings = self.dropout(embeddings)

RuntimeError: The size of tensor a (889) must match the size of tensor b (512) at non-singleton dimension 1

Where to_index =

 [('0015023cc06b5362d332b3baf348d11567ca2fbb',
  'The RNA pseudoknots in foot-and-mouth disease virus are dispensable for genome replication but essential for the production of infectious virus. 2 3\nword count: 194 22 Text word count: 5168 23 24 25 author/funder. All rights reserved. No reuse allowed without permission. Abstract 27 The positive stranded RNA genomes of picornaviruses comprise a single large open reading 28 frame flanked by 5′ and 3′ untranslated regions (UTRs). Foot-and-mouth disease virus (FMDV) 29 has an unusually large 5′ UTR (1.3 kb) containing five structural domains. These include the 30 internal ribosome entry site (IRES), which facilitates initiation of translation, and the cis-acting 31 replication element (cre). Less well characterised structures are a 5′ terminal 360 nucleotide 32 stem-loop, a variable length poly-C-tract of approximately 100-200 nucleotides and a series of 33 two to four tandemly repeated pseudoknots (PKs). We investigated the structures of the PKs 34 by selective 2′ hydroxyl acetylation analysed by primer extension (SHAPE) analysis and 35 determined their contribution to genome replication by mutation and deletion experiments. 36 SHAPE and mutation experiments confirmed the importance of the previously predicted PK 37 structures for their function. Deletion experiments showed that although PKs are not essential 38',
  None),
 ('00340eea543336d54adda18236424de6a5e91c9d',
  'Analysis Title: Regaining perspective on SARS-CoV-2 molecular tracing and its implications\nDuring the past three months, a new coronavirus (SARS-CoV-2) epidemic has been growing exponentially, affecting over 100 thousand people worldwide, and causing enormous distress to economies and societies of affected countries. A plethora of analyses based on viral sequences has already been published, in scientific journals as well as through non-peer reviewed channels, to investigate SARS-CoV-2 genetic heterogeneity and spatiotemporal dissemination. We examined all full genome sequences currently available to assess the presence of sufficient information for reliable phylogenetic and phylogeographic studies. Our analysis clearly shows severe limitations in the present data, in light of which any finding should be considered, at the very best, preliminary and hypothesis-generating. Hence the need for avoiding stigmatization based on partial information, and for continuing concerted efforts to increase number and quality of the sequences required for robust tracing of the epidemic.',
  None),
 ('004f0f8bb66cf446678dc13cf2701feec4f36d76',
  'Healthcare-resource-adjusted vulnerabilities towards the 2019-nCoV epidemic across China\n',
  None), ...]

How do I fix this? I don't see anywhere in the documentation about this. I assume the error message:

Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. 
Default to no truncation.

is related, and I need to set a max_length to 512 so that any documents that are larger than 512 get truncated to 512 tokens, but I don't see anywhere how to do that...

I'm getting the following error, when indexing images using POST to the txtai service url http://txtai.default.127.0.0.1.sslip.io/add.

"detail":[{"loc":["body"],"msg":"value is not a valid list","type":"type_error.list"}]}

Possible related to the FastAPI endpoint?

The same cluster is successful with text documents, but unsure how to index images.

Is it possible to periodically index images in a remote s3 directory via a workflow?

My current workflow YAML is:

writable: true
path: /tmp/index.tar.gz
cloud:
  provider: s3
  container: index
  key: "<key>"
  secret: "<secret>"
  host: txtai.s3.amazonaws.com
  port: 80

embeddings:
  path: "sentence-transformers/clip-ViT-B-32-multilingual-v1"
  content: true

I'm hoping to implement the Images embedding search in a workflow configuration, as in the examples/images.ipynb notebook

First of all, thank you for adding this feature!

I am attempting to recreate the example locally from the docs:

import numpy as np
import requests
from txtai.embeddings import Embeddings

data = ['test', 'test2']
def transform(inputs):
  response = requests.post("https://api-inference.huggingface.co/pipeline/feature-extraction/sentence-transformers/nli-mpnet-base-v2",
                           json={"inputs": inputs})

  return np.array(response.json(), dtype=np.float32)

# Index data using vectors from Inference API
embeddings = Embeddings({"method": "external", "transform": transform, "content": True})
embeddings.index([(uid, text, None) for uid, text in enumerate(data)])

print("%-20s %s" % ("Query", "Best Match"))
print("-" * 50)

print(embeddings.search('test', 1))

when I run the search, I receive an empty array. Perhaps data must be a tuple?

I have a lot of test data and embedding data /add-ed and /upsert-ed in a system, In that system when I do /search request

(with an SQL query like Select id,text, score, data from txtai where similar("what is the specialtiy of this ?",'8000') AND something ='something' ... )

it takes around 0 to 3 seconds to pick 10 results. I'm ok with that(because it may take time since the number data I have stored is larger like thousands of text documents' paragraphs extracted and stored ).

I think only /search API or /batchsearch API is gonna look into the 1000s of paragraphs to find the response. But /similarity API is not like them... (please correct me if I'm wrong) Because when we use http://localhost:8000/similarity we ourselves giving the list of texts and question in the request body.. and txtai is gonna return the list of id,score pairs only.

What I'm trying to say is, when I did the following (POST) request by port forwarding the txtAI from a Linux system that already has a lot of embeddings stored. It took 600ms to 1200ms to get the response (I checked this timing by sending requests few more times using Postman .)

curl -X POST "http://localhost:8000/similarity" -H "Content-Type: application/json" -d '{"query": "feel good story", "texts": ["Maine man wins $1M from $25 lottery ticket", "Dont sacrifice slower friends in a bear attack"]}'

But when I do the same in a fresh system or system with lesser embedding data, it is only taking 50ms or lesser than that to find the response.

Is there any relation between embedding data and /similarity API ?

(For a text list with 2 or 3 sentence, it is fine to take 600 to 1200 ms, but when I try to do the same with 100s of texts and 1 question.. it is taking 13 or more seconds)

My usecase is to do the /similarity operarion and calculate id,score for 100s of texts and 1 question in 1 to 3 seconds or lesser. I could achieve it in a system having lesser trained(embedding) data but not in the other case

So that's why I want to know, Is there any relation between embedding data and /similarity API ?

Actually I'm having a semantic search program in golang in which I'm using txtAI's APIs to perform searching, I'm making use of /search and /similarity APIs to fetch the results for the user query. Which is like after getting the 10 results from /search , I will be doing /similarity operation on each of 10 results's metadata keywords (that I have trained along with the "text" and "id" at the time of indexing them which will be in the "data" field ) following this approach I could improve the search result accuracy that I have expected, but the time it takes is more than 13 seconds when I have more keywords or tags in the "data" field.. I noticed that this time taken is only because of the /similarity API. Im pointing out the same in the following to make it clear to understand,

1. Do /search with a limit 10, Get 10 results and scores
2. Recalculate each result's score - 1 by 1 by.. finding /similarity between "user question"(query) and metadata content(texts) of each result - get the list of id and scores.
3. Merge these /similarity scores with /search scores and compute the scores again.
4. Now sort the results based on score again
5. Return 10 results to user.

Please suggest me an efficient way to improve the search speed and search accuracy.

I am trying to do a search using a query with the same limit as the input data length, but why is the output data length half of the input data length?

why len(result) < len(data)?

def get_embeddings(model_name, data):
    embeddings = Embeddings(
        {"path": model_name, "content": True, "objects": True})
    embeddings.index([(id, text, None) for id, text in enumerate(data)])
    return embeddings

def search(queries, embeddings, limit):
    return [result for result in embeddings.similarity(queries, limit)]

queries = (string)
data = [string, ...]
embeddings = get_embeddings(model_name, data)
result = search(queries, embeddings, len(data))

It seems like Cross Encoders are the preferred model for doing Re-Ranking of search results that were generated by another means (BM25, vector search etc...). However, if I provide one of these models as the path, all the results just have scores of 0.5.

Sentence Transformers recommends doing this. https://www.sbert.net/examples/applications/retrieve_rerank/README.html

In particular, their msmarco-minilm models seem ideal as a default (maybe the L-6 version?) https://www.sbert.net/docs/pretrained-models/ce-msmarco.html

Haystack's implementation uses this in its Ranker node. https://haystack.deepset.ai/pipeline_nodes/ranker

I've noticed when running ElasticSearch and txtai.pipeline for Similarity, the search (ranksearch) is very slow. When trying to search for 1 item, it can take upto 10 seconds.

The code I'm using is:

from txtai.pipeline import Similarity
from elasticsearch import Elasticsearch, helpers

# Connect to ES instance
es = Elasticsearch(hosts=["http://localhost:9200"], timeout=60, retry_on_timeout=True)

def ranksearch(query, limit):
  results = [text for _, text in search(query, limit * 10)]
  return [(score, results[x]) for x, score in similarity(query, results)][:limit]

def search(query, limit):
  query = {
      "size": limit,
      "query": {
          "query_string": {"query": query}
      }
  }

  results = []
  for result in es.search(index="articles", body=query)["hits"]["hits"]:
    source = result["_source"]
    results.append((min(result["_score"], 18) / 18, source["title"]))
  return results

similarity = Similarity("valhalla/distilbart-mnli-12-3")

limit = 1
query = "Bad News"
print(ranksearch(query, limit))

This branch adds one more type of trainer. This is the multi-label classification model. The edits are minor, but it was useful for me to train a multi_label classifier with txtai. I really appreciate txtai, let me know if you request any edits. Thanks

Is it possible to add new data in previously created index files ? E.g. If I have created and saved a Embedding Index file with 5K records and, after some days I have new 2K data which I want to include in my old index file then can I load the old index file and append the new 2K data into it so that when we query in the updated embedding index it will include the new 2K data too ? OR I need to re-index the whole 7K (old 5k + new 2K) data together ?

Minimal example below. I suspect something weird is happening to multithreading, but was not able to confirm or find a resolution.

Python version - 3.9.1
OS - Mac Big Sur

import os
import tempfile
import pandas as pd


from txtai.tokenizer import Tokenizer
from txtai.vectors import WordVectors
from txtai.embeddings import Embeddings

# generate random data
random_array = ["a" + pd.util.testing.rands_array(3, 2) for i in range(100)]
df = pd.DataFrame(random_array, columns=list("AB"))


with tempfile.NamedTemporaryFile(mode="w", suffix=".txt", delete=False) as output:
    tokens = output.name

    for row in range(df.shape[0]):
        output.write(" ".join(Tokenizer.tokenize(df.iloc[row, 0])) + "\n")

WordVectors.build(tokens, 10, 1, "search-data")

os.remove(tokens)

def stream():
    for row in range(df.shape[0]):
        uid = row
        tokens = Tokenizer.tokenize(df.iloc[row, 0])
        document = (uid, tokens, None)
        yield document

embeddings = Embeddings({"path": "search-data.magnitude"})

# Code hangs here
embeddings.index(stream())

I'd like to work with the actual numpy embeddings (the ones buffered here). Context on why later. I thought it was storevectors, but that looks to save the transformers model; and the embeddings file is the ANN index. I see two solutions: add a vector field to the SQL search query (eg, select id, text, vector from txtai) which will run faiss.reconstruct() here. I don't know how reconstruct works, but I see it used for that purpose in haystack. My concern is that this solution is lossy (does reconstruct try to regenerate the original vector?). Also you'd have to have an equivalent for the other ANNs. So I think a better approach is:

Add another config like save_actual_embeddings, which will copy over the tmpfile.npy. BUT, it would be hard to map the original IDs in - unless you did something dirty like the first column of the numpy array is the id (text, where the other columns are floats). I think a better approach is to save it as Pandas (or Feather / Arrow / whatever), with an ID column and vector column. I think this could be a simple add. But you could take it even further and scrap sqlite, just use the Pandas dataframe (wherein you can do sql-like queries, fields, etc) - two birds.

The context (feel free to skip). I'm building search. When they search, they get their search results like normal. But also, in the side bar there are a bunch of resources related to the results of their search. So it would take the vectors of all the returned results, np.mean() them, then send them to the various resource searches. Specifically it's a journal app, and a user will search through past entries. The result of that search will be say 10 entries. np.mean(10_entry_vectors) now goes to book-search, therapist-search, etc. So I don't want to search books/therapists from their query, but from their entries which result from their query. I also want to cluster their resultant entries (I still need to explore txtai's graph feature), for which I'd need the vectors). And finally (I'll submit a separate ticket, I'd like to search directly by vector, not text (using said mean).

Glad to know that txtAI brought Semantic graph as it's new feature. By the way how to actually use it if we have other language programmes and we expect it as an API ?.

Are there any API endpoints out like search and extract? Anybody using it ? Please let me know. It will be better(atleast for me) if we get the details on how to write the Python script into a yaml config file to use graphs, categories and topic modeling.

Following is my config file content to start the server using uvicorn..

# Index file path
path: ./tmp/index

# Allow indexing of documents
writable: True

# Enbeddings index
embeddings:
  path: sentence-transformers/all-MiniLM-L6-v2
  content: true
# I manually added these below lines upto extractor part
  functions:
  - name: graph
    function: graph.attribute
  expressions:
  - name: category
    expression: graph(indexid, 'category')
  - name: topic
    expression: graph(indexid, 'topic')
  - name: topicrank
    expression: graph(indexid, 'topicrank')
  graph:
    limit: 15
    minscore: 0.1
    topics:
      categories:
      - Society & Culture
      - Science & Mathematics
      - Health
      - Education & Reference
      - Computers & Internet
      - Sports
      - Business & Finance
      - Entertainment & Music
      - Family & Relationships
      - Politics & Government

extractor:
  path: distilbert-base-cased-distilled-squad

textractor:
    paragraphs: true
    minlength: 100
    join: false

Output:

...
ModuleNotFoundError: No module named 'graph' 
ERROR: Application startup failed. Exiting

After indexing/upserting around 200k images, I'm now seeing this exception about an SQLite constraint error. Any suggestions to recover?

    embeddings.upsert(images(x, y))
  File "/home/loop/apps/anaconda3/envs/pytorch/lib/python3.10/site-packages/txtai/embeddings/base.py", line 158, in upsert
    ids, _, embeddings = transform(documents, buffer)
  File "/home/loop/apps/anaconda3/envs/pytorch/lib/python3.10/site-packages/txtai/embeddings/transform.py", line 58, in __call__
    ids, dimensions, batches, stream = self.model.index(self.stream(documents), self.batch)
  File "/home/loop/apps/anaconda3/envs/pytorch/lib/python3.10/site-packages/txtai/vectors/base.py", line 79, in index
    for document in documents:
  File "/home/loop/apps/anaconda3/envs/pytorch/lib/python3.10/site-packages/txtai/embeddings/transform.py", line 112, in stream
    self.load(batch, offset)
  File "/home/loop/apps/anaconda3/envs/pytorch/lib/python3.10/site-packages/txtai/embeddings/transform.py", line 142, in load
    self.database.insert(batch, self.offset)
  File "/home/loop/apps/anaconda3/envs/pytorch/lib/python3.10/site-packages/txtai/database/sqlite.py", line 151, in insert
    self.insertsection(index, uid, document, tags, entry)
  File "/home/loop/apps/anaconda3/envs/pytorch/lib/python3.10/site-packages/txtai/database/sqlite.py", line 433, in insertsection
    self.cursor.execute(SQLite.INSERT_SECTION, [index, uid, text, tags, entry])
sqlite3.IntegrityError: UNIQUE constraint failed: sections.indexid

Exception ignored in: <generator object images at 0x7fe384949770>
RuntimeError: generator ignored GeneratorExit

Current config (embeddings.info() output):

{
  "backend": "faiss",
  "batch": 2,
  "build": {
    "create": "2022-11-24T23:38:28Z",
    "python": "3.10.6",
    "settings": {
      "components": "IDMap,Flat"
    },
    "system": "Linux (x86_64)",
    "txtai": "5.1.0"
  },
  "content": true,
  "dimensions": 512,
  "encodebatch": 2,
  "method": "sentence-transformers",
  "offset": 201487,
  "path": "clip-ViT-B-32",
  "update": "2022-11-25T03:38:44Z"
}

The translation pipeline seamlessly loads and uses a series of models to run the translations.

It would be beneficial to have a parameter to also return the associated models and detected languages to help with explainability and debugging.

This issue has re-emerged with torch 1.13. Same issue was reported in #300 with Python 3.7 and torch 1.12.

It appears Transformers is also having issues - https://github.com/huggingface/transformers/pull/19989

Pin build to torch==1.12.1 for now.

I was quite happy to see the new Cross Encoder support in the Similarity Pipeline given how well it has ranked in various tests - e.g. from this paper from Nils Reimer of SBERT and the SBERT documentation's recommendations for BM25 Retrieval + CE Re-Ranking. However, I'm finding it to be rather slow, even with a small number of documents from the initial BM25 step.

Nils subsequently published this paper about a new technique, Generative Pseudo Labelling (GPL), for unsupervised label generation for any text corpus which ultimately generates a fine-tuned embedding model.

It is nearly as relevant as BM25 + Cross Encoder Re-Ranking, particularly when combined with TSDAE (for "denoising"/precleaning text)

but it would be much faster at query-time than CE Re-Ranking (table from the first paper linked above - presumably would be similar to other embedding models, eg 200ms):

This would surely be expensive to pre-generate, but so are most dense vector embeddings and that's generally acceptable because it is done one time. The important part is the cross encoder-level relevance with normal bi-encoder speed.

I'm not sure what the best approach would be to incorporate into txtai, but it seems like TSDAE could be an optional pre-processing pipeline and then GPL could be an augmentation to the Labels pipeline (which seems to require you to pre-suggest labels rather than have them generated automatically and the HFTrainer pipeline.

And, perhaps, the GPL pipeline could be made up of individual pipeline components for the 3 steps:

1. Query generation
2. Negative Mining
3. Pseudo Labelling

(I'm not as sure about step 2 and 3, but the 1st step for Query Generation would certainly be a useful standalone Pipeline)