[crawler] jsonpath for data extraction

install

pip install jsonpath
#  perhaps 
pip install jsonpath -i https://pypi.tuna.tsinghua.edu.cn/simple

Use

from jsonpath import jsonpath
ret = jsonpath(json_dict, 'jsonpath Syntax rule string ')

jsonpath Rule of grammar

JSONPath	describe
$	Represents the root element
@	The current element
. or[]	Subelement
n/a	Parent element ,jsonpath Not supported
..	Regardless of location , Select the elements that meet the criteria
*	Match all element nodes
n/a	Access according to properties ,jsonpath Not supported , because json It's a key:value Recursive results of , No property access is required .
[]	Iterator marks , You can do simple iterations inside , Such as array subscript 、 Select the value according to the content .
[,]	Support iterators to make multiple choices
?()	Support filtering operation
()	Support expression evaluation
n/a	grouping ,jsonpath Not supported

give an example

from jsonpath import jsonpath
book_dict = {
     
  "store": {
    
    "book": [ 
      {
     "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      },
      {
     "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Sword of Honour",
        "price": 12.99
      },
      {
     "category": "fiction",
        "author": "Herman Melville",
        "title": "Moby Dick",
        "isbn": "0-553-21311-3",
        "price": 8.99
      },
      {
     "category": "fiction",
        "author": "J. R. R. Tolkien",
        "title": "The Lord of the Rings",
        "isbn": "0-395-19395-8",
        "price": 22.99
      }
    ],
    "bicycle": {
    
      "color": "red",
      "price": 19.95
    }
  }
}
res = jsonpath(book_dict, '$.store.book[*].author ')
print(res)
# Index from 0 Start  [ Indexes ]
res = jsonpath(book_dict , '$.store.book[0]')
print(res)
# Filter by criteria  ?(@. Field > The number )
# ..  Go directly to the selected field 
res = jsonpath(book_dict, '$..book[?(@.price>10)]')
print(res)

JSONPath	describe
$.store.book[*].author	obtain store All of the book The author of
$..author	Get all the authors
$.store.*	obtain store All elements under
$.store..price	obtain store The price of all books in
$..book[2]	Get the third book
$..book[(@.length-1)] | $..book[-1:]	Get the last book
$..book[0,1]| $..book[:2]	Get the first two books
$..book[?(@.isbn)]	The acquisition has isbn All of my books
$..book[?(@.price>10)]	Get a price greater than 10 All my books
$..*	Get all the data