Full text search for flask.

Overview

flask-msearch

https://img.shields.io/badge/pypi-v0.2.9-brightgreen.svg https://img.shields.io/badge/python-2/3-brightgreen.svg https://img.shields.io/badge/license-BSD-blue.svg

Installation

To install flask-msearch:

pip install flask-msearch
# when MSEARCH_BACKEND = "whoosh"
pip install whoosh blinker
# when MSEARCH_BACKEND = "elasticsearch", only for 6.x.x
pip install elasticsearch==6.3.1

Or alternatively, you can download the repository and install manually by doing:

git clone https://github.com/honmaple/flask-msearch
cd flask-msearch
python setup.py install

Quickstart

from flask_msearch import Search
[...]
search = Search()
search.init_app(app)

# models.py
class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content']

# views.py
@app.route("/search")
def w_search():
    keyword = request.args.get('keyword')
    results = Post.query.msearch(keyword,fields=['title'],limit=20).filter(...)
    # or
    results = Post.query.filter(...).msearch(keyword,fields=['title'],limit=20).filter(...)
    # elasticsearch
    keyword = "title:book AND content:read"
    # more syntax please visit https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
    results = Post.query.msearch(keyword,limit=20).filter(...)
    return ''

Config

# when backend is elasticsearch, MSEARCH_INDEX_NAME is unused
# flask-msearch will use table name as elasticsearch index name unless set __msearch_index__
MSEARCH_INDEX_NAME = 'msearch'
# simple,whoosh,elaticsearch, default is simple
MSEARCH_BACKEND = 'whoosh'
# table's primary key if you don't like to use id, or set __msearch_primary_key__ for special model
MSEARCH_PRIMARY_KEY = 'id'
# auto create or update index
MSEARCH_ENABLE = True
# logger level, default is logging.WARNING
MSEARCH_LOGGER = logging.DEBUG
# SQLALCHEMY_TRACK_MODIFICATIONS must be set to True when msearch auto index is enabled
SQLALCHEMY_TRACK_MODIFICATIONS = True
# when backend is elasticsearch
ELASTICSEARCH = {"hosts": ["127.0.0.1:9200"]}

Usage

from flask_msearch import Search
[...]
search = Search()
search.init_app(app)

class Post(db.Model):
    __tablename__ = 'basic_posts'
    __searchable__ = ['title', 'content']

    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(49))
    content = db.Column(db.Text)

    def __repr__(self):
        return '<Post:{}>'.format(self.title)

if raise sqlalchemy ValueError,please pass db param to Search

db = SQLalchemy()
search = Search(db=db)

Create_index

search.create_index()
search.create_index(Post)

Update_index

search.update_index()
search.update_index(Post)
# or
search.create_index(update=True)
search.create_index(Post, update=True)

Delete_index

search.delete_index()
search.delete_index(Post)
# or
search.create_index(delete=True)
search.create_index(Post, delete=True)

Custom Analyzer

only for whoosh backend

from jieba.analyse import ChineseAnalyzer
search = Search(analyzer=ChineseAnalyzer())

or use __msearch_analyzer__ for special model

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content', 'tag.name']
    __msearch_analyzer__ = ChineseAnalyzer()

Custom index name

If you want to set special index name for some model.

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content', 'tag.name']
    __msearch_index__ = "post111"

Custom schema

from whoosh.fields import ID

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content', 'tag.name']
    __msearch_schema__ = {'title': ID(stored=True, unique=True), 'content': 'text'}

Note: if you use hybrid_property, default field type is Text unless set special __msearch_schema__

Custom parser

from whoosh.qparser import MultifieldParser

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content']

    def _parser(fieldnames, schema, group, **kwargs):
        return MultifieldParser(fieldnames, schema, group=group, **kwargs)

    __msearch_parser__ = _parser

Note: Only for MSEARCH_BACKEND is whoosh

Custom index signal

flask-msearch uses flask signal to update index by default, if you want to use other asynchronous tools such as celey to update index, please set special MSEARCH_INDEX_SIGNAL

# app.py
app.config["MSEARCH_INDEX_SIGNAL"] = celery_signal
# or use string as variable
app.config["MSEARCH_INDEX_SIGNAL"] = "modulename.tasks.celery_signal"
search = Search(app)

# tasks.py
from flask_msearch.signal import default_signal

@celery.task(bind=True)
def celery_signal_task(self, backend, sender, changes):
    default_signal(backend, sender, changes)
    return str(self.request.id)

def celery_signal(backend, sender, changes):
    return celery_signal_task.delay(backend, sender, changes)

Relate index(Experimental)

for example

class Tag(db.Model):
    __tablename__ = 'tag'

    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(49))

class Post(db.Model):
    __tablename__ = 'post'
    __searchable__ = ['title', 'content', 'tag.name']

    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(49))
    content = db.Column(db.Text)

    # one to one
    tag_id = db.Column(db.Integer, db.ForeignKey('tag.id'))
    tag = db.relationship(
        Tag, backref=db.backref(
            'post', uselist=False), uselist=False)

    def __repr__(self):
        return '<Post:{}>'.format(self.title)

You must add msearch_FUN to Tag model,or the tag.name can’t auto update.

class Tag....
  ......
  def msearch_post_tag(self, delete=False):
      from sqlalchemy import text
      sql = text('select id from post where tag_id=' + str(self.id))
      return {
          'attrs': [{
              'id': str(i[0]),
              'tag.name': self.name
          } for i in db.engine.execute(sql)],
          '_index': Post
      }
Owner
honmaple
风落花语风落天,花落风雨花落田.
honmaple
PwnWiki Telegram database searching bot

pwtgbot PwnWiki Telegram database searching bot. Screenshots How it looks like in the terminal when running How it looks like in Telegram Run Directly

K4YT3X 3 Jan 25, 2022
A library for fast import of Windows NT Registry(REGF) into Elasticsearch.

A library for fast import of Windows NT Registry(REGF) into Elasticsearch.

S.Nakano 3 Apr 01, 2022
Inverted index creation and query search mechanism on Wikipedia pages.

WikiPedia Search Engine Step 1 : Installing Requirements Install "stemming" module for python using pip. Step 2 : Parsing the Data To parse the data,

Piyush Atri 1 Nov 27, 2021
solrpy is a Python client for Solr

solrpy solrpy is a Python client for Solr, an enterprise search server built on top of Lucene. solrpy allows you to add documents to a Solr instance,

Jiho Persy Lee 37 Jul 22, 2021
Whoosh indexing capabilities for Flask-SQLAlchemy, Python 3 compatibility fork.

Flask-WhooshAlchemy3 Whoosh indexing capabilities for Flask-SQLAlchemy, Python 3 compatibility fork. Performance improvements and suggestions are read

Blake VandeMerwe 27 Mar 10, 2022
Full-text multi-table search application for Django. Easy to install and use, with good performance.

django-watson django-watson is a fast multi-model full-text search plugin for Django. It is easy to install and use, and provides high quality search

Dave Hall 1.1k Jan 03, 2023
ForFinder is a search tool for folder and files

ForFinder is a search tool for folder and files. You can use that when you Source Code Analysis at your project's local files or other projects that you are download. Enter a root path and keyword to

Çağrı Aliş 7 Oct 25, 2022
An image inline search telegram bot.

Image-Search-Bot An image inline search telegram bot. Note: Use Telegram picture bot. That is better. Not recommending to deploy this bot. Made with P

Fayas Noushad 24 Oct 21, 2022
A web search server for ParlAI, including Blenderbot2.

Description A web search server for ParlAI, including Blenderbot2. Querying the server: The server reacting correctly: Uses html2text to strip the mar

Jules Gagnon-Marchand 119 Jan 06, 2023
Jina allows you to build deep learning-powered search-as-a-service in just minutes

Cloud-native neural search framework for any kind of data

Jina AI 17k Dec 31, 2022
esguard provides a Python decorator that waits for processing while monitoring the load of Elasticsearch.

esguard esguard provides a Python decorator that waits for processing while monitoring the load of Elasticsearch. Quick Start You need to launch elast

po3rin 5 Dec 08, 2021
ElasticSearch ODM (Object Document Mapper) for Python - pip install esengine

esengine - The Elasticsearch Object Document Mapper esengine is an ODM (Object Document Mapper) it maps Python classes in to Elasticsearch index/doc_t

SEEK International AI 109 Nov 22, 2022
Senginta is All in one Search Engine Scrapper for used by API or Python Module. It's Free!

Senginta is All in one Search Engine Scrapper. With traditional scrapping, Senginta can be powerful to get result from any Search Engine, and convert to Json. Now support only for Google Product Sear

33 Nov 21, 2022
GitScanner is a script to make it easy to search for Exposed Git through an advanced Google search.

GitScanner Legal disclaimer Usage of GitScanner for attacking targets without prior mutual consent is illegal. It is the end user's responsibility to

Kaio Gomes 3 Oct 28, 2022
Python Elasticsearch handler for the standard python logging framework

Python Elasticsearch Log handler This library provides an Elasticsearch logging appender compatible with the python standard logging library. This lib

Mohammed Mousa 0 Dec 08, 2021
Eland is a Python Elasticsearch client for exploring and analyzing data in Elasticsearch with a familiar Pandas-compatible API.

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

elastic 463 Dec 30, 2022
This project is a sample demo of Arxiv search related to AI/ML Papers built using Streamlit, sentence-transformers and Faiss.

This project is a sample demo of Arxiv search related to AI/ML Papers built using Streamlit, sentence-transformers and Faiss.

Karn Deb 49 Oct 30, 2022
A sentence search engine that fetches examples from trusted news/media organisations. Great for writing better English.

A sentence search engine that fetches examples from trusted news/media websites. Great for improving writing & speaking better English.

Stephen Appiah 1 Apr 04, 2022
Search emails from a domain through search engines

EmailFinder - search emails through Search Engines

Josué Encinar 155 Dec 30, 2022
Wagtail CLIP allows you to search your Wagtail images using natural language queries.

Wagtail CLIP allows you to search your Wagtail images using natural language queries.

Matt Segal 10 Dec 21, 2022