Convert HTML to Markdown-formatted text.

Related tags

Web Content Extractingpythonmarkdownmarkdown-parser
Overview

html2text

Build Status Coverage Status Downloads Version Wheel? Format License

html2text is a Python script that converts a page of HTML into clean, easy-to-read plain ASCII text. Better yet, that ASCII also happens to be valid Markdown (a text-to-HTML format).

Usage: html2text [filename [encoding]]

Option Description
--version Show program's version number and exit
-h, --help Show this help message and exit
--ignore-links Don't include any formatting for links
--escape-all Escape all special characters. Output is less readable, but avoids corner case formatting issues.
--reference-links Use reference links instead of links to create markdown
--mark-code Mark preformatted and code blocks with [code]...[/code]

For a complete list of options see the docs

Or you can use it from within Python:

>>> import html2text
>>>
>>> print(html2text.html2text("<p><strong>Zed's</strong> dead baby, <em>Zed's</em> dead.</p>"))
**Zed's** dead baby, _Zed's_ dead.

Or with some configuration options:

>>> import html2text
>>>
>>> h = html2text.HTML2Text()
>>> # Ignore converting links from HTML
>>> h.ignore_links = True
>>> print h.handle("<p>Hello, <a href='https://www.google.com/earth/'>world</a>!")
Hello, world!

>>> print(h.handle("<p>Hello, <a href='https://www.google.com/earth/'>world</a>!"))

Hello, world!

>>> # Don't Ignore links anymore, I like links
>>> h.ignore_links = False
>>> print(h.handle("<p>Hello, <a href='https://www.google.com/earth/'>world</a>!"))
Hello, [world](https://www.google.com/earth/)!

Originally written by Aaron Swartz. This code is distributed under the GPLv3.

How to install

html2text is available on pypi https://pypi.org/project/html2text/

$ pip install html2text

How to run unit tests

tox

To see the coverage results:

coverage html

then open the ./htmlcov/index.html file in your browser.

Documentation

Documentation lives here

Comments
  • 3.200.3 vs 2014.7.3 output quirks

    3.200.3 vs 2014.7.3 output quirks

    Just upgraded from 3.200.3 to 2014.7.3 and noticed the following things:

    Bold text inside links

    <a href="link.htm"><b>Text</b></a>

    Before: [**Text**](link.htm) After: **[Text**](link.htm) (to me this looks incorrect)

    Image links

    <a href="images/image.jpg"><img alt="Title" src="images/thumbnails/image.jpg"></img></a>

    Before: [![Title](images/thumbnails/image.jpg)](images/image.jpg) After: ![Title](images/thumbnails/image.jpg)

    Literal links

    Links like this [http://example.com](http://example.com) now look like this <http://example.com>. Is that valid markdown?

    Escapes

    A lot of unnecessary escapes: \--, 1\.

    Downgraded back to 3.200.3

    bug 
    opened by max-arnold 22
  • Malformed output of links

    Malformed output of links 

    >>> import html2text
>>> h = html2text.HTML2Text()
>>> h.handle('<a href="http://www.test.com">http://www.test.com</a>')  #  this fails
u'<http://www.test.com>\n\n'
>>> h.handle('<a href="http://www.test.com/">http://www.test.com</a>')  # adding slash works
u'[http://www.test.com](http://www.test.com/)\n\n'

    while this works as expected

    >>> h.handle('<a href="http://www.test.com/">test</a>')
u'[test](http://www.test.com/)\n\n'
>>> h.handle('<a href="http://www.test.com">test</a>')
u'[test](http://www.test.com)\n\n'
    bug 
    opened by barsch 16
  • Fix issue with emphasis and whitespace

    Fix issue with emphasis and whitespace

    This fixes some issues that occurs with white spaces around the following emphasis marks ~~, **, _. It's not the most beautiful code, but it fixes the bugs.

    opened by jonathan-s 15
  • unexpanded < > &

    unexpanded < > &

    From: https://bugs.debian.org/791470

    Version: 2015.6.21-1 (and current master):

    $ echo '<body>&lt;&gt;&amp;</body>' | html2markdown
&lt;&gt;&amp;

    It worked correctly in 2014.9.25-1:

    $ echo '<body>&lt;&gt;&amp;</body>' | html2markdown
<>&
    bug 
    opened by stefanor 15
  • escaping surrogate.

    escaping surrogate.

    Throwing errors for my command:

    curl http://www.baeldung.com/websockets-spring|html2text|vim -

    Traceback (most recent call last): File "C:\Users\mohi\AppData\Local\Programs\Python\Python36\Scripts\html2text-script.py", line 11, in load_entry_point('html2text==2017.10.4', 'console_scripts', 'html2text')() File "c:\users\mohi\appdata\local\programs\python\python36\lib\site-packages\html2text\cli.py", line 306, in main wrapwrite(h.handle(data)) File "c:\users\mohi\appdata\local\programs\python\python36\lib\site-packages\html2text\utils.py", line 207, in wrapwrite text = text.encode('utf-8') UnicodeEncodeError: 'utf-8' codec can't encode character '\udc9d' in position 676: surrogates not allowed

    opened by ahmedmohiduet 14
  • Don't split paragraphs in blockquotes

    Don't split paragraphs in blockquotes

    This PR should fix #139. When fixing this issue I discovered that whitespace plays a crucial role at the end of the line which is rather annoying. This PR also partially fixes this as it removes whitespace from some lines.

    enhancement 
    opened by jonathan-s 13
  • UnicodeDecode Error

    UnicodeDecode Error

    I am trying to use html2text to clean up html tags on news reports scraped from Google RSS feed. I run into some UnicodeDecode errors. Specifically, I run html2text directly on the command line html2text --ignore-links --ignore-images 52778881361118.htm > test.txt I could not enclose the file as the issue tracker won't take such files. It gives me the following error: Traceback (most recent call last): File "/usr/local/bin/html2text", line 8, in load_entry_point('html2text==2014.9.25', 'console_scripts', 'html2text')() File "/Library/Python/2.7/site-packages/html2text/init.py", line 1083, in main data = data.decode(encoding) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0xa5 in position 22495: invalid start byte

    Do you have any insight into how I can work around this? Thanks, Philippe

    invalid 
    opened by ploustaunau 13
  • Wraps long URLs

    Wraps long URLs

    Forwarding aaronsw/html2text#7, so it doesn't get forgotten: Forwarding http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=616090:

    Long URLs are wrapped, which they probably shouldn't be.

    Example:

    <html>
<head><title>Test</title></head>
<body>
<p>And <a href="http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=multiarch;[email protected]">here</a> is a long link I had at hand.</p>
</body>
</html>

    Results in:

    And [here][1] is a long link I had at hand.

   [1]: http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=multiarch;users
[email protected]
    bug 
    opened by stefanor 13
  • Add callback feature

    Add callback feature

    This PR enables finer-grained control over the output generated. Example usage:

    my_em = False
def my_tag_handle(parser, tag, attrs, start):
  global my_em
  if tag == 'em':
    if 'class' in attrs and 'my' in attrs['class'] and start:
      parser.o("[my] ")
      my_em = True
      return True
    elif my_em and not start:
      parser.o("[/my]")
      my_em = False
      return True

parser = html2text.HTML2Text()
parser.tag_callback = my_tag_handle
text = parser.handle(html)
    enhancement 
    opened by critiqjo 12
  • Long links wrapping option

    Long links wrapping option

    Possible fix for #38

    • urls which are long are not wrapped if so desired.
    • old behaviour is maintained to avoid breaking someone's code
    • --no-wrap-links has been added.
    • both inline links and reference links are supported for no wrapping.
    • docs have been updated
    opened by theSage21 12
  • Emtpy link title of <a><img></a> with images_to_alt

    Emtpy link title of with images_to_alt 

    # Python 3.4.3 (default, Aug 10 2015, 16:40:44) 
# [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
>>> import html2text
>>> html2text.__version__
(2016, 4, 2)

>>> txt = html2text.HTML2Text()
>>> txt.images_to_alt = True
>>> txt.handle('<a href="http://google.com"><img src="images/google.png"></a>')
'[](http://google.com)\n\n'

    So the conversion:

    <a href="http://google.com"><img src="images/google.png"></a>

    [](http://google.com)\n\n

    Seem to come up with an empty tile. I would like to get an output like

    [image](http://google.com)\n\n

    or be able to ignore such links completely. Is there a way yet to archive that?

    opened by luckydonald 11
  • HTML <picture> Element not returned as image link from srcset

    HTML Element not returned as image link from srcset

    The image link from the srcset is not returned in the markdown return in the <picture> html element. I expect it to be returned like if the image src was in the <img> html element.

    Code snippet example:

    import html2text

html = """
<section>
    <h1>Poorly drawn lines comics</h1>
    <picture>
        <source
            sizes="(min-resolution: 4dppx) and (max-width: 700px) 50vw, (-webkit-min-device-pixel-ratio: 4) and (max-width: 700px) 50vw, (min-resolution: 3dppx) and (max-width: 700px) 67vw, (-webkit-min-device-pixel-ratio: 3) and (max-width: 700px) 65vw, (min-resolution: 2.5dppx) and (max-width: 700px) 80vw, (-webkit-min-device-pixel-ratio: 2.5) and (max-width: 700px) 80vw, (min-resolution: 2dppx) and (max-width: 700px) 100vw, (-webkit-min-device-pixel-ratio: 2) and (max-width: 700px) 100vw, 700px"
            srcset=" https://pbs.twimg.com/media/FbVo3fiUcAAYytB?format=jpg&name=smal 640w, 
                https://pbs.twimg.com/media/FbVo3fiUcAAYytB?format=jpg&name=medium 828w, 
                https://pbs.twimg.com/media/FbVo3fiUcAAYytB?format=jpg&name=large 1400w" />
        <img alt="" />
    </picture>
    <p>
        This is one of my most favorite recent comics. Comes in print too. I want it for my home.
    </p>
</section>
"""
md = html2text.html2text(html)
print(md)

    Actual Output:

    # Poorly drawn lines comics

This is one of my most favorite recent comics. Comes in print too. I want it
for my home.

    Expected Output:

    • includes the image link (though I'm not particular for which one)
    • same result as if using the <img> html element
    # Poorly drawn lines comics

![](https://pbs.twimg.com/media/FbVo3fiUcAAYytB?format=jpg&name=small)

This is one of my most favorite recent comics. Comes in print too. I want it
for my home.
    • Version by html2text --version 2020.1.16
    • Python version python --version 3.9.13
    opened by contendaClara 0
  • Support Python 3.10

    Support Python 3.10

    • Support Python 3.10 (edits to .travis.yml, setup.cfg and tox.ini files)

    In the process of implementing the above, this fix was required:

    • Fix test data for br_inside_a (my creation, not sure how this wrong version got pushed and passed Travis CI)
    • Setting environment variable PYTHONUTF8 = 1 in tox.ini as Unicode tests would fail due to encoding defaulting to windows-1252 on Windows
    • Added .gitattribute to have correct line endings

    In the process of fixing the above, the following was discovered:

    • Reversed order of arguments in assertions in tests (was assert result == actual but pytest standard -- and IDESs -- expect the reverse order, i.e. assert actual == expected)

    In the process of all the above the following was in the way:

    • Modern IDEs don't allow *.md files to end with multiple empty lines and will automatically trim them out, so I had to modify tests to assert actual.rstrip() == expected.rstrip()

    All tox tests are passed by this PR.

    opened by mborsetti 0
  • Link titles break with encoded quote

    Link titles break with encoded quote

    • Version by html2text --version: 2020.1.16
    • Test script:
    $ printf '<a title="&quot;" href="/">foo</a>' | html2text
[foo](/ """)

    [foo](/ """) → [foo](/ """) [foo](/ "\"") → foo

    • Python version python --version: 3.10.5
    opened by xPMo 0
  • --ignore-links flag creates new composite words in output

    --ignore-links flag creates new composite words in output

    Hi! I'm doing some natural language processing experiments and using html2text to make text sources out of internet pages. My problem is words in links are sticking to each other if I use --ignore-link flag:

    html2text --ignore-links <<< '<a href="/1">1</a><a href="/2">2</a><a href="/3">3 4</a><a href="/5">5</a>'
123 45

    example is specially simplified of course, but it "creates" new composite words. I've patched it locally to add spaces after each ignored link, sort of workaround with minimal changes:

    if tag == "a" and self.ignore_links and not start:
    self.o(" ")
if tag == "a" and not self.ignore_links:

    and it produces what I need:

    html2text --ignore-links <<< '<a href="/1">1</a><a href="/2">2</a><a href="/3">3 4</a><a href="/5">5</a>'
1 2 3 4 5

    Should I open a pull request for this? The code above is sort of workaround, but if it will be useful - I'd be happy to make it cleaner, add tests, changelog, etc.

    • Version by html2text --version: 2020.1.16 (from pypi, but github/master version is affected too)
    • Python version python3.8 --version: Python 3.8.0`
    opened by strizhechenko 0
Releases(2019.8.11)

  • 2019.8.11(Aug 11, 2019)

    • Add support for wrapping list items.
    • Fix #201: handle ‎/‏ marks mid-text within stressed tags or right after stressed tags.
    • Feature #213: images_as_html config option to always generate an img html tag. preserves "height", "width" and "alt" if possible.
    • Remove support for end-of-life Pythons. Now requires Python 2.7 or 3.4+.
    • Remove support for retrieving HTML over the network.
    • Add __main__.py module to allow running the CLI using python -m html2text ....
    • Fix #238: correct spacing when a HTML entity follows a non-stressed tags which follow a stressed tag.
    • Remove unused or deprecated:
      • html2text.compat.escape()
      • html2text.config.RE_UNESCAPE
      • html2text.HTML2Text.replaceEntities()
      • html2text.HTML2Text.unescape()
      • html2text.unescape()
    • Fix #208: handle LEFT-TO-RIGHT MARK after a stressed tag.

    PyPI: https://pypi.org/project/html2text/2019.8.11/

    Source code(tar.gz)
    Source code(zip)

  • 2018.1.9(Jan 10, 2018)

    2018.9.1

    • Fix #188: Non-ASCII in title attribute causes encode error.
    • Feature #194: Add support for the tag.
    • Feature #193: Add support for the tag.

    PyPI: https://pypi.python.org/pypi/html2text/2018.1.9

    Source code(tar.gz)
    Source code(zip)

  • 2017.10.4(Oct 4, 2017)

    Version 2017.10.4

    • Fix #157: Fix images link with div wrap
    • Fix #55: Fix error when empty title tags
    • Fix #160: The html2text tests are failing on Windows and on Cygwin due to differences in eol handling between windows/*nix
    • Feature #164: Housekeeping: Add flake8 to the travis build, cleanup existing flake8 violations, add py3.6 and pypy3 to the travis build
    • Fix #109: Fix for unexpanded < > &
    • Fix #143: Fix line wrapping for the lines starting with bold
    • Adds support for numeric bold text indication in font-weight, as used by Google (and presumably others.)
    • Fix #173 and #142: Stripping whitespace in crucial markdown and adding whitespace as necessary
    • Don't drop any cell data on tables uneven row lengths (e.g. colspan in use)

    PyPI: https://pypi.python.org/pypi/html2text/2017.10.4

    Source code(tar.gz)
    Source code(zip)

  • 2016.9.19(Sep 20, 2016)

    2016.9.19

    • Default image alt text option created and set to a default of empty string "" to maintain backward compatibility
    • Fix #136: --default-image-alt now takes a string as argument
    • Fix #113: Stop changing quiet levels on /script tags.
    • Merge #126: Fix deprecation warning on py3 due to html.escape
    • Fix #145: Running test suite on Travis CI for Python 2.6.

    PyPI: https://pypi.python.org/pypi/html2text/2016.9.19

    Source code(tar.gz)
    Source code(zip)

  • 2016.5.29(May 29, 2016)

    2016.5.29

    2016.5.29

    • Fix #125: --pad_tables now pads table cells to make them look nice.
    • Fix #114: Break does not interrupt blockquotes
    • Deprecation warnings for URL retrieval.

    PyPI: https://pypi.python.org/pypi/html2text/2016.5.29

    Source code(tar.gz)
    Source code(zip)

  • 2016.4.2(Apr 1, 2016)

  • 2016.1.8(Jan 8, 2016)

    2016.1.8

    • Feature #99: Removed duplicated initialization.
    • Fix #100: Get element style key error.
    • Fix #101: Fix error end tag pop exception.
    • <s>, <strike>, <del> now rendered as ~~text~~.

    PyPi: https://pypi.python.org/pypi/html2text/2016.1.8

    Source code(tar.gz)
    Source code(zip)

  • 2015.11.4(Nov 4, 2015)

    2015.11.4

    • Fix #38: Long links wrapping controlled by --no-wrap-links.
    • Note: --no-wrap-links implies --reference-links
    • Feature #83: Add callback-on-tag.
    • Fix #87: Decode errors can be handled via command line.
    • Feature #95: Docs, decode errors spelling mistake.
    • Fix #84: Make bodywidth kwarg overridable using config.

    PyPi: https://pypi.python.org/pypi/html2text/2015.11.4

    Source code(tar.gz)
    Source code(zip)

  • 2015.6.21(Jun 21, 2015)

    2015.6.21

    • Fix #31: HTML entities stay inside link.
    • Fix #71: Coverage detects command line tests.
    • Fix #39: Documentation update.
    • Fix #61: Functionality added for optional use of automatic links.
    • Feature #80: title attribute is preserved in both inline and reference links.
    • Feature #82: More command line options. See docs.

    Pypi: https://pypi.python.org/pypi/html2text/2015.6.21

    Source code(tar.gz)
    Source code(zip)

  • 2015.6.12(Jun 12, 2015)

    2015.6.12

    • Feature #76: Making pre blocks clearer for further automatic formatting.
    • Fix #71: Coverage detects tests carried out in subprocesses

    PyPi: https://pypi.python.org/pypi/html2text/2015.6.12

    Source code(tar.gz)
    Source code(zip)

  • 2015.6.6(Jun 5, 2015)

    2015.6.6

    • Fix #24: 3.200.3 vs 2014.7.3 output quirks.
    • Fix #61. Malformed links in markdown output.
    • Feature #62: Automatic version number.
    • Fix #63: Nested code, anchor bug.
    • Fix #64: Proper handling of anchors with content that starts with tags.
    • Feature #67: Documentation all over the module.
    • Feature #70: Adding tests for the module.
    • Fix #73: Typo in config documentation.
    Source code(tar.gz)
    Source code(zip)

  • 2015.4.14(Apr 14, 2015)

    2015.4.14

    • Feature #59: Write image tags with height and width attrs as raw html to retain dimensions.

    PyPi: https://pypi.python.org/pypi/html2text/2015.4.14

    Source code(tar.gz)
    Source code(zip)

  • 2015.4.13(Apr 13, 2015)

    2015.4.13

    • Feature #56: Treat '-' file parameter as stdin.
    • Feature #57: Retain escaping of html except within code or pre tags.

    PyPi: https://pypi.python.org/pypi/html2text/2015.4.13

    Source code(tar.gz)
    Source code(zip)

  • 2015.2.18(Feb 18, 2015)

  • 2014.12.29(Dec 29, 2014)

    2014.12.29

    • Feature #51: Add single line break option. This feature is useful for ensuring that lots of extra line breaks do not end up in the resulting Markdown file in situations like Evernote .enex exports. Note that this only works properly if body-width is set to 0.

    PyPi: https://pypi.python.org/pypi/html2text/2014.12.29

    Source code(tar.gz)
    Source code(zip)

  • 2014.12.24(Dec 24, 2014)

    2014.12.24

    • Feature #49: Added a images_to_alt option to discard images and keep only their alt.
    • Feature #50: Protect links, surrounding them with angle brackets to avoid breaking...
    • Feature: Add setup.cfg file.

    PyPi: https://pypi.python.org/pypi/html2text/2014.12.24

    Source code(tar.gz)
    Source code(zip)

  • 2014.12.5(Dec 5, 2014)

    2014.12.5

    • Feature: Update README.md with usage examples.
    • Fix #35: Remove py_modules from setup.py.
    • Fix #36: Excludes tests from being installed as a separate module.
    • Fix #37: Don't hardcode the path to the installed binary.
    • Fix: Readme typo in running cli.
    • Feature #40: Extract cli part to cli module.
    • Feature #42: Bring python version compatibility to compat.py module.
    • Feature #41: Extract utility/helper methods to utils module.
    • Fix #45: Does not accept standard input when running under Python 3.
    • Feature: Clean up ChangeLog.rst for version and date numbers.

    PyPi: https://pypi.python.org/pypi/html2text/2014.12.5

    Source code(tar.gz)
    Source code(zip)

  • 2014.9.25(Sep 25, 2014)

    2014.9.25 - 2014-09-25

    • Feature #29, #27: Add simple table support with bypass option.
    • Fix #20: Replace project website with: http://alir3z4.github.io/html2text/ .
    Source code(tar.gz)
    Source code(zip)

  • 2014.9.8(Sep 18, 2014)

  • 2014.9.7(Sep 18, 2014)

    2014.9.7 - 2014-09-07

    • Fix unicode/type error in memory leak unit-test.
    • Feature #16: Remove install_deps.py.
    • Feature #17: Add status badges via pypin.
    • Feature #18: Add Python 3.4 to travis config file.
    • Feature #19: Bring html2text to a separate module and take out the conf/constant variables.
    • Feature #21: Remove meta vars from html2text.py file header.
    • Fix: Fix TypeError when parsing tags like . Fixed in #25.
    Source code(tar.gz)
    Source code(zip)

  • 2014.7.3(Jul 4, 2014)

    2014.7.3 - 2014-07-03

    Fix #8: Remove How to do a release section from README.md. Fix #11: Include test directory markdown, html files. Fix #13: memory leak in using handle while keeping the old instance of html2text.

    PyPi: https://pypi.python.org/pypi/html2text/2014.7.3

    Source code(tar.gz)
    Source code(zip)

  • 2014.4.5(Jul 4, 2014)

    2014.4.5 - 2014-04-05

    Fix #1: Add ChangeLog.rst file. Fix #2: Add AUTHORS.rst file.

    PyPi: https://pypi.python.org/pypi/html2text/2014.4.5

    Source code(tar.gz)
    Source code(zip)
Owner
Alireza Savand
I am Alireza Savand, a Software Architect.
Alireza Savand
GitHub Repository alir3z4.github.io/html2text/
Open clone of OpenAI's unreleased WebText dataset scraper.

Open clone of OpenAI's unreleased WebText dataset scraper. This version uses pushshift.io files instead of the API for speed.

Joshua C Peterson 471 Dec 30, 2022
Export your data from Xiami

Xiami Exporter 导出虾米音乐的个人数据，功能： 导出歌曲为 json 收藏歌曲 收藏专辑 播放列表 导出收藏艺人为 json 导出收藏专辑为 json 导出播放列表为 json (个人和收藏) 将导出的数据整理至 sqlite 数据库 收藏歌曲 收藏艺人 收藏专辑 播放列表 下载已导出

Xiao Meng 59 Nov 13, 2021
Zotero2Readwise - A Python Library to retrieve annotations and notes from Zotero and upload them to your Readwise

Zotero ➡️ Readwise zotero2readwise is a Python library that retrieves all Zotero

Essi Alizadeh 49 Dec 20, 2022
Module for automatic summarization of text documents and HTML pages.

Automatic text summarizer Simple library and command line utility for extracting summary from HTML pages or plain texts. The package also contains sim

Mišo Belica 3k Jan 03, 2023
Web Content Retrieval for Humans™

Lassie Lassie is a Python library for retrieving basic content from websites. Usage import lassie lassie.fetch('http://www.youtube.com/watch?v

Mike Helmick 571 Dec 29, 2022
a small library for extracting rich content from urls

A small library for extracting rich content from urls. what does it do? micawber supplies a few methods for retrieving rich metadata about a variety o

Charles Leifer 588 Dec 27, 2022
News, full-text, and article metadata extraction in Python 3. Advanced docs:

Newspaper3k: Article scraping & curation Inspired by requests for its simplicity and powered by lxml for its speed: "Newspaper is an amazing python li

Lucas Ou-Yang 12.3k Jan 01, 2023
Github Actions采集RSS, 打造无广告内容优质的头版头条超赞宝藏页

Github Actions Rss (garss, 嘎RSS! 已收集69个RSS源, 生成时间: 2021-02-26 11:23:45) 信息茧房是指人们关注的信息领域会习惯性地被自己的兴趣所引导，从而将自己的生活桎梏于像蚕茧一般的“茧房”中的现象。

zhaoolee 721 Jan 02, 2023
Convert HTML to Markdown-formatted text.

html2text html2text is a Python script that converts a page of HTML into clean, easy-to-read plain ASCII text. Better yet, that ASCII also happens to

Alireza Savand 1.3k Dec 31, 2022
Fast and robust date extraction from web pages, with Python or on the command-line

Find original and updated publication dates of any web page. From the command-line or within Python, all the steps needed from web page download to HTML parsing, scraping, and text analysis are inclu

Adrien Barbaresi 60 Dec 14, 2022
fast python port of arc90's readability tool, updated to match latest readability.js!

python-readability Given a html document, it pulls out the main body text and cleans it up. This is a python port of a ruby port of arc90's readabilit

Yuri Baburov 2.2k Dec 28, 2022
RSS feed generator website with user friendly interface

RSS feed generator website with user friendly interface

Alexandr Nesterenko 331 Jan 02, 2023
Every web site provides APIs.

Toapi Overview Toapi give you the ability to make every web site provides APIs. Version v2.0.0, Completely rewrote. More elegant. More pythonic v1.0.0

Jiuli Gao 3.3k Jan 05, 2023
Web-Extractor - Simple Tool To Extract IP-Adress From Website

IP-Adress Extractor Simple Tool To Extract IP-Adress From Website Socials: Langu

ميخائيل 7 Jan 16, 2022
Pythonic HTML Parsing for Humans™

Requests-HTML: HTML Parsing for Humans™ This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible. When us

Python Software Foundation 12.9k Jan 01, 2023
Combine XPath, CSS Selectors and JSONPath for Web data extracting.

Data Extractor Combine XPath, CSS Selectors and JSONPath for Web data extracting. Quickstarts Installation Install the stable version from PYPI. pip i

林玮 (Jade Lin) 27 Oct 22, 2022
Brownant is a web data extracting framework.

Brownant Brownant is a lightweight web data extracting framework. Who uses it? At the moment, dongxi.douban.com (a.k.a. Douban Dongxi) uses Brownant i

Douban Inc. 157 Jan 06, 2022