Abstract

Currently ZeroNet uses a wrapper for sidebar and notifications UI, and embeds site content with an iframe. This proposal fixes some issues caused by iframing by embedding ZeroNet UI into site content.

Rationale

• #2165, #2057 - All features have to be marked unsafe by default, so supporting new ones is troublesome;
• #2154 - Most people don't understand what NOSANDBOX means;
• #2086 - Some interactive content in iframes might get blocked;
• #2058 - Pull-to-refresh is working weird with iframes;
• #2003 - "Opening as child window" is sometimes impossible to bypass;
• #1736 - Opening link in a new tab is more difficult than required;
• #1707, #565 - Embedding a site into a site is unsafe with an iframe;
• #1667 - Browser header style can only be set by root frame;
• #1403 - Scrolling is impossible, iframe is most likely the reason;
• #1399, #1054, #29 - localStorage, IndexedDB, etc. are not available in sandboxed iframes;
• #1237 - Fullscreen API doesn't work;
• #1236, #1262 - Some Vue plugins don't work;
• #1010 - Creating Web Workers requires data: URI usage;
• #469 - History management requires ZeroFrame usage.

Overview

Current architecture:

┌─────────────────────────────────────────────────────┐
│ WRAPPER (src/Ui/template/wrapper.html)              │
│ Stores secrets (e.g. wrapper key)                   │
│ ┌─────────────────────────────────────────────────┐ │
│ │ WS UPLINK                                       │ │
│ │ UiWebsocket API                                 │ │
│ └─────────────────────────────────────────────────┘ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ NOTIFICATIONS (<div>)                           │ │
│ │ ╔═════════════════════════════════════════════╗ │ │
│ │ ║ SITE NOTIFICATIONS (<div>)                  ║ │ │
│ │ ║ Unsafe content (possible XSS)               ║ │ │
│ │ ╚═════════════════════════════════════════════╝ │ │
│ │ ┌─────────────────────────────────────────────┐ │ │
│ │ │ WRAPPER NOTIFICATIONS (<div>)               │ │ │
│ │ │ Safe content (no leaking or spoofing)       │ │ │
│ │ └─────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────┘ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ SIDEBAR (<div>)                                 │ │
│ │ Site and key management, safe                   │ │
│ │ ╔═════════════════════════════════════════════╗ │ │
│ │ ║ SITE DATA                                   ║ │ │
│ │ ║ Title, description, donate links, etc.      ║ │ │
│ │ ╚═════════════════════════════════════════════╝ │ │
│ └─────────────────────────────────────────────────┘ │
│ ╔═════════════════════════════════════════════════╗ │
│ ║ IFRAME SANDBOX                                  ║ │
│ ║ Unsafe content (managed by site code)           ║ │
│ ╚═════════════════════════════════════════════════╝ │
└─────────────────────────────────────────────────────┘

Safety layers are separated with double border.

No software has zero bugs. This includes security issues. There are many possible attack points here:

• Site notifications (protected by manual escaping) were exploited by me, revealing site private keys;
• Sidebar content (protected by manual escaping) wasn't exploited according to my sources, but it's still a valid point of attack;
• Iframe sandbox (protected by browser) is in theory unbreakable, but still adds unnecessary complexity;
• The final attack point is the external iframe (in case the wrapper is in an iframe itself). This was exploited by me once.

Proposed architecture;

┌─────────────────────────────────────────────────────┐
│ HTML PAGE                                           │
│ ┌─────────────────────────────────────────────────┐ │
│ │ PREFIX (shadow DOM)                             │ │
│ │ Practically invisible to site content           │ │
│ │ ┌─────────────────────────────────────────────┐ │ │
│ │ │ SITE NOTIFICATIONS (<div>)                  │ │ │
│ │ │ Unsafe content (possible XSS)               │ │ │
│ │ └─────────────────────────────────────────────┘ │ │
│ │ ╔═════════════════════════════════════════════╗ │ │
│ │ ║ WRAPPER NOTIFICATIONS (<iframe>)            ║ │ │
│ │ ║ Safe content (no leaking or spoofing)       ║ │ │
│ │ ╚═════════════════════════════════════════════╝ │ │
│ │ ╔═════════════════════════════════════════════╗ │ │
│ │ ║ SIDEBAR (iframe)                            ║ │ │
│ │ ║ Site and key management                     ║ │ │
│ │ ║ ╔═════════════════════════════════════════╗ ║ │ │
│ │ ║ ║ GATE (iframe)                           ║ ║ │ │
│ │ ║ ║ ┌─────────────────────────────────────┐ ║ ║ │ │
│ │ ║ ║ │ WS UPLINK                           │ ║ ║ │ │
│ │ ║ ║ │ UiWebsocket API (ADMIN)             │ ║ ║ │ │
│ │ ║ ║ └─────────────────────────────────────┘ ║ ║ │ │
│ │ ║ ╚═════════════════════════════════════════╝ ║ │ │
│ │ ╚═════════════════════════════════════════════╝ │ │
│ │ ╔═════════════════════════════════════════════╗ │ │
│ │ ║ GATE (iframe)                               ║ │ │
│ │ ║ ┌─────────────────────────────────────────┐ ║ │ │
│ │ ║ │ WS UPLINK                               │ ║ │ │
│ │ ║ │ UiWebsocket API (site)                  │ ║ │ │
│ │ ║ └─────────────────────────────────────────┘ ║ │ │
│ │ ╚═════════════════════════════════════════════╝ │ │
│ └─────────────────────────────────────────────────┘ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ SITE DATA                                       │ │
│ │ Unsafe content (managed by site code)           │ │
│ └─────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘

Sure, it might look more difficult at the first glance, but security comes with a cost.

Resulting HTML

The resulting HTML (the one that browser receives) consists of a prefix and the real site .html file.

Prefix

Prefix is a "magic" HTML code that sets up an analogue of what was a wrapper by creating a shadow DOM node. This ensures that the sidebar and notifications are shown correctly, independent of main site styles, and that it doesn't affect the site itself.

Gate

A gate is an iframe that acts like a gate between UiWebsocket and its user. The gate handler uses the referrer to check what permissions the websocket must have.

Fixed issues / added features

• #2165, #2057 - All features are now safe;
• #2154 - No need for NOSANDBOX anymore;
• #2086 - The only interactive content is simple notifications (i.e. buttons);
• #2058 - Pull-to-refresh can now be controlled by the site;
• #2003 - No need to be top frame anymore;
• #1736 - Opening in a new tab made easy;
• #1707, #565 - Embedding made possible (requires a separate proposal);
• #1667 - Controlled by the site now;
• #1403 - Scrolling is fixed;
• #1399, #1054, #29 - localStorage, IndexedDB, etc. made possible (requires a separate proposal);
• #1237 - Fullscreen API without ZeroFrame;
• #1236, #1262 - SecurityError's fixed;
• #1010 - Web Workers by path should work now;
• #469 - History API without ZeroFrame.

Implementation status

• [x] Prefix
• [x] Gate
• [ ] 0 button
• [ ] Sidebar
• [ ] Debug logs
• [ ] Notifications
• [ ] Modified panel
• [x] Old wrapper compatiblity
• [ ] Local storage
• [ ] Wrapper commands
  • [x] innerReady [compat]
  • [x] innerLoaded & wrapperInnerLoaded [compat]
  • [ ] wrapperNotification, wrapperConfirm, wrapperPrompt, wrapperProgress
  • [x] wrapperSetViewport [compat]
  • [x] wrapperSetTitle [compat]
  • [x] wrapperReload [compat]
  • [ ] wrapperGetLocalStorage & wrapperSetLocalStorage
  • [x] wrapperPushState, wrapperReplaceState, wrapperGetState [compat]
  • [ ] wrapperGetAjaxKey
  • [x] wrapperOpenWindow [compat]
  • [ ] wrapperPermissionAdd
  • [x] wrapperRequestFullscreen [compat]
  • [ ] wrapperWebNotification & wrapperCloseWebNotification [compat]

ETA: at most 1 week if no major issues are found

sslcrypto is a new MIT-licensed library I've recently made. It supports AES and ECC. I originally made it to get rid of GPLv3 pyelliptic, but interestingly it turned out even more useful.

First, it replaces pyelliptic, pybitcointools and a part of CryptMessage so there's less code involved. Second, it supports both OpenSSL backend and pure-Python fallback both for AES and ECC (pyelliptic doesn't support pure-Python ECC so CryptMessage didn't work on some machines). Third, it's licensed under MIT (I can add any other license you choose) so we wouldn't have problems if we wanted to re-license a part of ZeroNet code under some lax license. Fourth, it supports all OpenSSL versions since 0.9.8 (other cryptography libraries support a lot less). Fifth, its fallback implementation is a bit safer than pybitcointools because it fixes a vulnerability (I don't want to name it here to avoid possible private key leaks).

Sixth, it's fast. Plain performance tests say that it's about as fast as pyelliptic (both for AES and ECC) and that its fallback implementation is 1.5 times faster than pybitcointools (ECIES only, ECDSA is about the same). However, some tests I ran on ZeroMail show that message decryption is about 2 or 3 times faster than the current version (I've only run tests on single-thread model because more threads are slower for me for some reason both for sslcrypto and your version).

Additionally, sslcrypto is tested on Python 3.4 to 3.8 with OpenSSL 0.9.8, 1.0.0, 1.0.1, 1.0.2, 1.1.0 and 1.1.1.

/cc @filips123

This PR closes #1974, closes #917 and closes #993.

If you can not wait the official ipv6 support, you can use ipv6 version directly replacing the core code (see: zihu.bit/?Topic:1537342516_1K7tW4WEbso38uMs4RN3W7GihA6tTqrApz/+IPV6) (And I delete the original source in github because zeronet would transfer to Python3 and many codes would change, this PR would be just a reference for Python3.) IPV6 Supports: (If you donot have ipv6, these change would not change the logic of your code. If you have public ipv6, these change would let you choose ipv6 as your connection server first (can let the users to choose if needed ), not your public ipv4 address, in which you can also connect these ipv4 peers and ipv4 trackers. )

• [x] UDP/TCP Trackers Supports

• [x] Peer Exchange Supports

• [x] BigFile Supports

• [x] Bootstrapper Supports

• [x] Tested in real condition

Ok, so first we have our license violations. All of these have a license that's currently incompatible with ZeroNet's GPLv2 License:

• Sidebar/media_globe/globe.js - Apache 2
• pyelliptic - GPLv3
• msgpack - Apache 2
• rsa - Apache 2
• gevent websocket - Apache 2
• bencode.py - BitTorrent License
• Coincurve - Apache 2
• python-bitcoinlib

Now... I propose 2 steps that will solve all of this:

1. Switch to GPLv3 license. This will allow for pyelliptic, python-bitcoinlib, and all Apache 2 licensed-libraries being used (Apache 2 is not compatible with GPLv2, but is compatible with GPLv3)
2. Switch out bencode.py for an alternative that @imachug has just published to PyPi - https://pypi.org/project/bencode-open/

If you don't wish to change to GPLv3, then we need to find a way to replace all of the 8 libraries above, or we can switch out pyelliptic and python-bitcoinlib for an alternative and find a new license that is compatible with Apache 2.

The Merger sites feature allow to to query and display other site's data. Using this its possible to create infinitely scalable social sites by having separate sites for every user profile, so you will get updates for profiles that you follow.

ZeroMe: A twitter-like social site

• [x] Layout mockups:
  • [x] Welcome page
  • [x] Profile profile
  • [x] User search
  • [x] News feed
• [x] Static html version
  • [x] Welcome page
  • [x] Profile profile
  • [x] User search
  • [x] News feed
• [x] Logo
• [X] Merger site permission handling/request
• [X] New database structure to allow easier and faster joins
• [X] Merge sub-site data files to merger site's database
• [X] Sub-site adding/delete/list
• [x] Make Sign/File oprations merger site compatible: actionSiteSign, actionSitePublish, actionFileWrite, actionFileDelete, actionFileGet, actionFileRules
• [x] Profile page data structure
• [x] Merger site rebuild DB
• [x] Profile page db structure
• [x] User directory data structure
• [x] User directory db structure
• [x] Update database on new merger/merged site add/remove
• [x] User directory
• [x] Sub-site feed listing
• [x] Profile creation
• [x] Profile page
• [x] Profile editing
• [x] Posting
• [x] Post editing
• [x] Post deleting
• [x] Activity list
• [x] Like
• [x] Commenting
• [x] Comment editing
• [x] Comment deleting
• [x] Follow profile
• [x] List followed users
• [x] Auto download new site on follow
• [x] Following without registered profile
• [x] Also update user database on profile modification
• [x] User content delete solution for efficient user directory archiving
• [x] Avatar upload

Later:

• [ ] Re-share
• [ ] File upload
• [ ] Image upload with thumbnail generation (optional files)

ZeroHello:

• [ ] Group sites by type

ZeroTalk: http://127.0.0.1:43110/Talk.ZeroNetwork.bit/?Topic:1538407463_14ytAKDfNjArMTqGecTi7ginG3aZTsRAum/Rebase+ZeroNet+infrastructure+to+IPFS+or+Dat

Advantage: DHT, archiving system, bigger developer team & community, no repeated code… Also a chance to migrate to Python 3.

Before I start to write code, is there any suggestion for me?

I have registered a domain with the .bit tld and configured the name accordingly, also included in the content.json file the domain field but can't get it work!

Ping: @HelloZeroNet @JeremyRand @imachug

This is the first PR of the "Road to better sandbox" series. This one announces lower-case site addresses as well as full site addresses to trackers.

The next steps should be:

• Refer to sites with lowercase addresses only;
• Support http://address.zeronet or zhttp://address or zero://address schemes;
• Switch to reverse iframe.

Possibilities:

• Browser plugin with custom zero:// protocol handler. Access sites: zero://talk.zeronetwork.bit zero://1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr
• Browser plugin, but keep http://. Access sites: http://1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr(.zero?), http://talk.zeronetwork.bit. Can be problematic if .bit site also has zeronet and clearnet address
• Using system's host file maps all site address to 127.0.0.0, this allows to access zeronet sites in any browser using http://talk.zeronetwork.bit (needs to map zeronet webui to port 80). Access new sites can be problematic.
• Register a domain and map all subdomain to 127.0.0.1 (needs to map zeronet webui to port 80). Access sites using http://talk.zeronetwork.bit.gozero.net
• Extend zeronet to also act as dns server and configure your os to 127.0.0.1 dns. Access sites like http://talk.zeronetwork.bit

Something you can do now: add 127.0.0.1 zero to your host file, start zeronet with zeronet.py --ui_port 80 then you can access zeronet sites http://zero/talk.zeronetwork.bit

Probably the browser plugin is the best solution (if possible)

IPV6 Supports: (If you donot have ipv6, these change would not change the logic of your code. If you have public ipv6, these change would let you choose ipv6 as your connection server first (can let the users to choose if needed ), not your public ipv4 address, in which you can also connect these ipv4 peers and ipv4 trackers. )

• [x] UDP/TCP Trackers Supports

• [x] Peer Exchange Supports

• [x] BigFile Supports

• [x] Bootstrapper Supports

• [x] Testing

To reduce space/network bw used by data files:

• On network protocol level: less work, but does not reduces the storage requirements
• On file storage level

@caryoscelus is still spamming this repository with her ostensibly "zeronet conservancy" fork, which is nothing more than a scam in which she simply changed the donation addresses without making any significant changes.

You should avoid using the fork promoted by @caryoscelus; she has also spammed the ZeroNet Wikipedia page with links to her fork.

While the name implies "preserving ZeroNet," the entire fork is about defrauding ZeroNet users and "rewriting ZeroNet" from Python to a creepy language called Rust, which was created in 2010. She formed a phoney organization called "Riza committee" with some seriously deranged individuals with the goal of completely destroying ZeroNet.

Except for the word "conservation," "ZeroNet conservancy" refers to everything. It's a giant scam.

You're mistaken if you think you can hide behind @imachug.

Imachug is a retard who attacked ZeroNet users and faked a license change, literally manipulating legal voting to ensure that ZeroNet will have a GPLv3 license rather than a GPLv3+, then @imachug pretended he is standing with RMS (Richard Stallman) and even became the creator of the "Stand with RMS" campaign on GitHub, which has since been archived.

If, as he claims, @imachug has changed over the years, he should speak out against your hostile fork @caryoscelus. Even if he doesn't, I will, so prepare yourself.

I will not fail ZeroNet; I did not fail when the license was changed, and I will not fail now.

What happened？

There are 1 security vulnerabilities found in msgpack 0.4.4

• MPS-2022-14994

What did I do？

Upgrade msgpack from 0.4.4 to 0.6.0 for vulnerability fix

What did you expect to happen？

Ideally, no insecure libs should be used.

The specification of the pull request

PR Specification from OSCS

Step 1: Please describe your environment

• ZeroNet version: _____
• Operating system: _____
• Web browser: _____
• Tor status: not available/always/disabled
• Opened port: yes/no
• Special configuration: ____

Step 2: Describe the problem:

Steps to reproduce:

Observed Results:

• What happened? This could be a screenshot, a description, log output (you can send log/debug.log file to [email protected] if necessary), etc.

Expected Results:

• What did you expect to happen?

Step 1: Vulnerable Code to SQL INJECTION

• ZeroNet version: latest
• Operating system: all operating system

Step 2: Describe the problem:

The types of attacks that can be performed using SQL injection vary depending on the type of database engine. The attack works on dynamic SQL statements. A dynamic statement is a statement that is generated at run time using parameters password from a web form or URI query string. Vulnerable Code: https://github.com/HelloZeroNet/ZeroNet/blob/py3/plugins/OptionalManager/ContentDbPlugin.py line 126 is vulnerable to sql injection res = cur.execute("SELECT * FROM content WHERE size_files_optional > 0 AND site_id = %s" % site_id) an attacker can bypass like that "site_id OR malicious_sql_query#

How to prevent SQLI?

User input should never be trusted – It must always be sanitized before it is used in dynamic SQL statements. Stored procedures – these can encapsulate the SQL statements and treat all input as parameters. Prepared statements –prepared statements to work by creating the SQL statement first then treating all submitted user data as parameters. This has no effect on the syntax of the SQL statement. Regular expressions –these can be used to detect potential harmful code and remove it before executing the SQL statements. Database connection user access rights –only necessary access rights should be given to accounts used to connect to the database. This can help reduce what the SQL statements can perform on the server. Error messages –these should not reveal sensitive information and where exactly an error occurred. Simple custom error messages such as “Sorry, we are experiencing technical errors. The technical team has been contacted. Please try again later” can be used instead of display the SQL statements that caused the error.

Observed Results:

Expected Results:

• SQL injection attack occurs when:

An unintended data enters a program from an untrusted source. The data is used to dynamically construct a SQL query

错误：./main.go：177：8：标识符中的字符 U+FF1A '：' 无效 错误：./main.go：177：8： 语法错误： 意外： 在语句的末尾 错误：./main.go：177：19： 无效字符 U+3002 '。' 在标识符中 错误：./main.go：177：31：标识符中的字符 U+FF08 '（' 无效 错误：./main.go：177：34： 标识符中的字符 U+201C '“' 无效 错误：./main.go：177：38： 标识符中的字符 U+201D '“' 无效 错误：./main.go：177：41：标识符中的字符 U+FF0C '，' 无效 错误：./main.go：177：44： 标识符中的字符 U+201C '“' 无效 错误：./main.go：177：56： 标识符中的字符 U+201D '“' 无效 错误：进程已完成，退出代码为 2。