When trying to install using python3.6, I see:

/usr/pkg/bin/python3.6  viewvc-install --prefix=/usr/pkg/share/viewvc --destdir=/scratch/www/viewvc/work/.destdir)
  File "viewvc-install", line 50
    ("bin/cgi/viewvc.cgi",        "bin/cgi/viewvc.cgi",        0755, 1, 0, 0),
                                                                  ^
SyntaxError: invalid token
*** Error code 1

Please add support for python3 here, and in other places, if necessary. Thanks!

Description

On systems with repository roots whose directory names contain non-ASCII
characters (say, for example, a UTF-8 Japanese string), ViewVC will fail to
correctly convert the path into UTF-8 as required by Subversion's API.

Metadata Imported from Tigris (Issue 500)

• Creation Date: 2011-11-14 11:13:31
• Reporter: cmpilato
• Subcomponent: core
• Version: 1.1.12
• Milestone: unscheduled
• Keywords: patch
• Cc:

Comments

2011-11-14 11:16:25 by cmpilato

Created an attachment (id=256)
Not-fully-tested patch for this issue

2011-11-14 14:06:59 by cmpilato

Created an attachment (id=257)
Not-fully-tested patch for this issue (v2)

2011-11-28 08:49:03 by cmpilato

This would be less tedious, I think, if 'viewvc-install' (which is less likely
to be running in a "C"-locale shell) would query the system's locale and
populate the appropriate configuration bits.

Attachments

issue-500-patch.txt - Not-fully-tested patch for this issue

Posted 2011-11-14 11:16:25 by cmpilato

Fix issue #500 ("ViewVC chokes on non-ASCII Subversion repository root
name").

* conf/viewvc.conf.dist
  (general.locale): New.

* lib/config.py
  (Config.set_defaults): Set default value for general.locale (to None).

* bin/asp/query.asp,
* bin/asp/viewvc.asp,
* bin/cgi/query.cgi,
* bin/cgi/viewvc.cgi,
* bin/cvsdbadmin,
* bin/loginfo-handler,
* bin/mod_python/query.py,
* bin/mod_python/viewvc.py,
* bin/standalone.py,
* bin/svndbadmin,
* bin/wsgi/query.wsgi,
* bin/wsgi/viewvc.fcgi,
* bin/wsgi/viewvc.wsgi
  After fetching the Config object, call locale.setlocale() to dictate
  any configured locale.
--This line, and those below, will be ignored--

Index: conf/viewvc.conf.dist
===================================================================
--- conf/viewvc.conf.dist   (revision 2670)
+++ conf/viewvc.conf.dist   (working copy)
@@ -197,6 +197,17 @@
 ##
 #address =

+## locale: Specifies the server's system locale.  This helps ViewVC
+## know how to interact with your server, for example when it needs to
+## convert between its internal path encoding and the encoding used on
+## your system.
+##
+## Examples:
+## locale = en_US.UTF-8
+## locale = Russian_Russia.1251
+##
+#locale = 
+
 ## kv_files: Provides a mechanism for custom key/value pairs to be
 ## available to templates.  These are stored in key/value (KV) files.
 ##
Index: lib/config.py
===================================================================
--- lib/config.py   (revision 2670)
+++ lib/config.py   (working copy)
@@ -382,6 +382,7 @@
     self.general.address = ''
     self.general.kv_files = [ ]
     self.general.languages = ['en-us']
+    self.general.locale = None

     self.utilities.rcs_dir = ''
     if sys.platform == "win32":
Index: bin/standalone.py
===================================================================
--- bin/standalone.py   (revision 2670)
+++ bin/standalone.py   (working copy)
@@ -422,6 +422,9 @@
 def handle_config(config_file):
   global cfg
   cfg = viewvc.load_config(config_file or CONF_PATHNAME)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)


 def usage():
Index: bin/mod_python/viewvc.py
===================================================================
--- bin/mod_python/viewvc.py    (revision 2670)
+++ bin/mod_python/viewvc.py    (working copy)
@@ -55,6 +55,9 @@
 def index(req):
   server = sapi.ModPythonServer(req)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   try:
     viewvc.main(server, cfg)
   finally:
Index: bin/mod_python/query.py
===================================================================
--- bin/mod_python/query.py (revision 2670)
+++ bin/mod_python/query.py (working copy)
@@ -61,6 +61,9 @@
     fp.close()

 cfg = viewvc.load_config(CONF_PATHNAME)
+if cfg.general.locale is not None:
+  import locale
+  locale.setlocale(locale.LC_ALL, cfg.general.locale)

 def index(req):
   server = sapi.ModPythonServer(req)
Index: bin/asp/viewvc.asp
===================================================================
--- bin/asp/viewvc.asp  (revision 2670)
+++ bin/asp/viewvc.asp  (working copy)
@@ -58,6 +58,9 @@
 server = sapi.AspServer(Server, Request, Response, Application)
 try:
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc.main(server, cfg)
 finally:
   s.close()
Index: bin/asp/query.asp
===================================================================
--- bin/asp/query.asp   (revision 2670)
+++ bin/asp/query.asp   (working copy)
@@ -54,6 +54,9 @@
 server = sapi.AspServer(Server, Request, Response, Application)
 try:
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc_base_url = cfg.query.viewvc_base_url
   if viewvc_base_url is None:
     viewvc_base_url = "viewvc.asp"
Index: bin/svndbadmin
===================================================================
--- bin/svndbadmin  (revision 2670)
+++ bin/svndbadmin  (working copy)
@@ -239,6 +239,9 @@

 def main(command, repository, revs=[], verbose=0, force=0):
     cfg = viewvc.load_config(CONF_PATHNAME)
+    if cfg.general.locale is not None:
+        import locale
+        locale.setlocale(locale.LC_ALL, cfg.general.locale)
     db = cvsdb.ConnectDatabase(cfg)

     # Purge what must be purged.
Index: bin/cgi/viewvc.cgi
===================================================================
--- bin/cgi/viewvc.cgi  (revision 2670)
+++ bin/cgi/viewvc.cgi  (working copy)
@@ -58,4 +58,7 @@

 server = sapi.CgiServer()
 cfg = viewvc.load_config(CONF_PATHNAME, server)
+if cfg.general.locale is not None:
+  import locale
+  locale.setlocale(locale.LC_ALL, cfg.general.locale)
 viewvc.main(server, cfg)
Index: bin/cgi/query.cgi
===================================================================
--- bin/cgi/query.cgi   (revision 2670)
+++ bin/cgi/query.cgi   (working copy)
@@ -54,6 +54,9 @@

 server = sapi.CgiServer()
 cfg = viewvc.load_config(CONF_PATHNAME, server)
+if cfg.general.locale is not None:
+  import locale
+  locale.setlocale(locale.LC_ALL, cfg.general.locale)
 viewvc_base_url = cfg.query.viewvc_base_url
 if viewvc_base_url is None:
   viewvc_base_url = "viewvc.cgi"
Index: bin/loginfo-handler
===================================================================
--- bin/loginfo-handler (revision 2670)
+++ bin/loginfo-handler (working copy)
@@ -221,6 +221,9 @@

 def ProcessLoginfo(rootpath, directory, files):
     cfg = viewvc.load_config(CONF_PATHNAME)
+    if cfg.general.locale is not None:
+        import locale
+        locale.setlocale(locale.LC_ALL, cfg.general.locale)
     db = cvsdb.ConnectDatabase(cfg)
     repository = vclib.ccvs.CVSRepository(None, rootpath, None,
                                           cfg.utilities, 0)
Index: bin/wsgi/viewvc.fcgi
===================================================================
--- bin/wsgi/viewvc.fcgi    (revision 2670)
+++ bin/wsgi/viewvc.fcgi    (working copy)
@@ -44,6 +44,9 @@
 def application(environ, start_response):
   server = sapi.WsgiServer(environ, start_response)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc.main(server, cfg)
   return []

Index: bin/wsgi/viewvc.wsgi
===================================================================
--- bin/wsgi/viewvc.wsgi    (revision 2670)
+++ bin/wsgi/viewvc.wsgi    (working copy)
@@ -37,5 +37,8 @@
 def application(environ, start_response):
   server = sapi.WsgiServer(environ, start_response)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc.main(server, cfg)
   return []
Index: bin/wsgi/query.wsgi
===================================================================
--- bin/wsgi/query.wsgi (revision 2670)
+++ bin/wsgi/query.wsgi (working copy)
@@ -38,6 +38,9 @@
 def application(environ, start_response):
   server = sapi.WsgiServer(environ, start_response)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc_base_url = cfg.query.viewvc_base_url
   if viewvc_base_url is None:
     viewvc_base_url = "viewvc.wsgi"
Index: bin/cvsdbadmin
===================================================================
--- bin/cvsdbadmin  (revision 2670)
+++ bin/cvsdbadmin  (working copy)
@@ -171,6 +171,9 @@
     rootpath = vclib.ccvs.canonicalize_rootpath(root)
     try:
         cfg = viewvc.load_config(CONF_PATHNAME)
+        if cfg.general.locale is not None:
+            import locale
+            locale.setlocale(locale.LC_ALL, cfg.general.locale)
         db = cvsdb.ConnectDatabase(cfg)

         if command in ('rebuild', 'purge'):

issue-500-patch.2.txt - Not-fully-tested patch for this issue (v2)

Posted 2011-11-14 14:06:58 by cmpilato

Fix issue #500 ("ViewVC chokes on non-ASCII Subversion repository root
name").

* conf/viewvc.conf.dist
  (general.locale): New.

* lib/config.py
  (Config.set_defaults): Set default value for general.locale (to None).

* bin/asp/query.asp,
* bin/asp/viewvc.asp,
* bin/cgi/query.cgi,
* bin/cgi/viewvc.cgi,
* bin/cvsdbadmin,
* bin/loginfo-handler,
* bin/mod_python/query.py,
* bin/mod_python/viewvc.py,
* bin/standalone.py,
* bin/svndbadmin,
* bin/wsgi/query.wsgi,
* bin/wsgi/viewvc.fcgi,
* bin/wsgi/viewvc.wsgi
  After fetching the Config object, call locale.setlocale() to dictate
  any configured locale.
--This line, and those below, will be ignored--

Index: conf/viewvc.conf.dist
===================================================================
--- conf/viewvc.conf.dist   (revision 2670)
+++ conf/viewvc.conf.dist   (working copy)
@@ -197,6 +197,17 @@
 ##
 #address =

+## locale: Specifies the server's system locale.  This helps ViewVC
+## know how to interact with your server, for example when it needs to
+## convert between its internal path encoding and the encoding used on
+## your system.
+##
+## Examples:
+## locale = en_US.UTF-8
+## locale = Russian_Russia.1251
+##
+#locale = 
+
 ## kv_files: Provides a mechanism for custom key/value pairs to be
 ## available to templates.  These are stored in key/value (KV) files.
 ##
Index: lib/config.py
===================================================================
--- lib/config.py   (revision 2670)
+++ lib/config.py   (working copy)
@@ -382,6 +382,7 @@
     self.general.address = ''
     self.general.kv_files = [ ]
     self.general.languages = ['en-us']
+    self.general.locale = None

     self.utilities.rcs_dir = ''
     if sys.platform == "win32":
Index: bin/standalone.py
===================================================================
--- bin/standalone.py   (revision 2670)
+++ bin/standalone.py   (working copy)
@@ -422,6 +422,9 @@
 def handle_config(config_file):
   global cfg
   cfg = viewvc.load_config(config_file or CONF_PATHNAME)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)


 def usage():
Index: bin/mod_python/viewvc.py
===================================================================
--- bin/mod_python/viewvc.py    (revision 2670)
+++ bin/mod_python/viewvc.py    (working copy)
@@ -55,6 +55,9 @@
 def index(req):
   server = sapi.ModPythonServer(req)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   try:
     viewvc.main(server, cfg)
   finally:
Index: bin/mod_python/query.py
===================================================================
--- bin/mod_python/query.py (revision 2670)
+++ bin/mod_python/query.py (working copy)
@@ -61,6 +61,9 @@
     fp.close()

 cfg = viewvc.load_config(CONF_PATHNAME)
+if cfg.general.locale is not None:
+  import locale
+  locale.setlocale(locale.LC_ALL, cfg.general.locale)

 def index(req):
   server = sapi.ModPythonServer(req)
Index: bin/asp/viewvc.asp
===================================================================
--- bin/asp/viewvc.asp  (revision 2670)
+++ bin/asp/viewvc.asp  (working copy)
@@ -58,6 +58,9 @@
 server = sapi.AspServer(Server, Request, Response, Application)
 try:
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc.main(server, cfg)
 finally:
   s.close()
Index: bin/asp/query.asp
===================================================================
--- bin/asp/query.asp   (revision 2670)
+++ bin/asp/query.asp   (working copy)
@@ -54,6 +54,9 @@
 server = sapi.AspServer(Server, Request, Response, Application)
 try:
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc_base_url = cfg.query.viewvc_base_url
   if viewvc_base_url is None:
     viewvc_base_url = "viewvc.asp"
Index: bin/svndbadmin
===================================================================
--- bin/svndbadmin  (revision 2670)
+++ bin/svndbadmin  (working copy)
@@ -239,6 +239,9 @@

 def main(command, repository, revs=[], verbose=0, force=0):
     cfg = viewvc.load_config(CONF_PATHNAME)
+    if cfg.general.locale is not None:
+        import locale
+        locale.setlocale(locale.LC_ALL, cfg.general.locale)
     db = cvsdb.ConnectDatabase(cfg)

     # Purge what must be purged.
Index: bin/cgi/viewvc.cgi
===================================================================
--- bin/cgi/viewvc.cgi  (revision 2670)
+++ bin/cgi/viewvc.cgi  (working copy)
@@ -58,4 +58,7 @@

 server = sapi.CgiServer()
 cfg = viewvc.load_config(CONF_PATHNAME, server)
+if cfg.general.locale is not None:
+  import locale
+  locale.setlocale(locale.LC_ALL, cfg.general.locale)
 viewvc.main(server, cfg)
Index: bin/cgi/query.cgi
===================================================================
--- bin/cgi/query.cgi   (revision 2670)
+++ bin/cgi/query.cgi   (working copy)
@@ -54,6 +54,9 @@

 server = sapi.CgiServer()
 cfg = viewvc.load_config(CONF_PATHNAME, server)
+if cfg.general.locale is not None:
+  import locale
+  locale.setlocale(locale.LC_ALL, cfg.general.locale)
 viewvc_base_url = cfg.query.viewvc_base_url
 if viewvc_base_url is None:
   viewvc_base_url = "viewvc.cgi"
Index: bin/loginfo-handler
===================================================================
--- bin/loginfo-handler (revision 2670)
+++ bin/loginfo-handler (working copy)
@@ -221,6 +221,9 @@

 def ProcessLoginfo(rootpath, directory, files):
     cfg = viewvc.load_config(CONF_PATHNAME)
+    if cfg.general.locale is not None:
+        import locale
+        locale.setlocale(locale.LC_ALL, cfg.general.locale)
     db = cvsdb.ConnectDatabase(cfg)
     repository = vclib.ccvs.CVSRepository(None, rootpath, None,
                                           cfg.utilities, 0)
Index: bin/wsgi/viewvc.fcgi
===================================================================
--- bin/wsgi/viewvc.fcgi    (revision 2670)
+++ bin/wsgi/viewvc.fcgi    (working copy)
@@ -44,6 +44,9 @@
 def application(environ, start_response):
   server = sapi.WsgiServer(environ, start_response)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc.main(server, cfg)
   return []

Index: bin/wsgi/query.fcgi
===================================================================
--- bin/wsgi/query.fcgi (revision 2670)
+++ bin/wsgi/query.fcgi (working copy)
@@ -45,6 +45,9 @@
 def application(environ, start_response):
   server = sapi.WsgiServer(environ, start_response)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc_base_url = cfg.query.viewvc_base_url
   if viewvc_base_url is None:
     viewvc_base_url = "viewvc.fcgi"
Index: bin/wsgi/viewvc.wsgi
===================================================================
--- bin/wsgi/viewvc.wsgi    (revision 2670)
+++ bin/wsgi/viewvc.wsgi    (working copy)
@@ -37,5 +37,8 @@
 def application(environ, start_response):
   server = sapi.WsgiServer(environ, start_response)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc.main(server, cfg)
   return []
Index: bin/wsgi/query.wsgi
===================================================================
--- bin/wsgi/query.wsgi (revision 2670)
+++ bin/wsgi/query.wsgi (working copy)
@@ -38,6 +38,9 @@
 def application(environ, start_response):
   server = sapi.WsgiServer(environ, start_response)
   cfg = viewvc.load_config(CONF_PATHNAME, server)
+  if cfg.general.locale is not None:
+    import locale
+    locale.setlocale(locale.LC_ALL, cfg.general.locale)
   viewvc_base_url = cfg.query.viewvc_base_url
   if viewvc_base_url is None:
     viewvc_base_url = "viewvc.wsgi"
Index: bin/cvsdbadmin
===================================================================
--- bin/cvsdbadmin  (revision 2670)
+++ bin/cvsdbadmin  (working copy)
@@ -171,6 +171,9 @@
     rootpath = vclib.ccvs.canonicalize_rootpath(root)
     try:
         cfg = viewvc.load_config(CONF_PATHNAME)
+        if cfg.general.locale is not None:
+            import locale
+            locale.setlocale(locale.LC_ALL, cfg.general.locale)
         db = cvsdb.ConnectDatabase(cfg)

         if command in ('rebuild', 'purge'):

On subversion repository, PATH_INFO is always UTF-8, however on CVS repository, arbitrary nul terminated C string can be used for path. So we should use bytes or Unicode string with 'surrogateescape' as PATH_INFO, at least before which repository is pointed by the path is determined.

For WSGI, this was implemented except it is using 'replace' error handler, so this PR change to use 'surrogateescape' handler.

For CGI, PATH_INFO wasn't decoded, this add a code to do it.

Edit 2: looks like the exceptions are thrown only when hr_intraline is set.

Recently, we upgraded a Subversion installation to 1.14.1 on a CentOS 7 server.

From what we can tell, Subversion 1.14.1 no longer has bindings for Python 2 meaning we had to switch to the development branch ("master") of ViewVC 1.3.0 at commit f97ea61 of Dec 1, 2021.

Everything appears to be working except the "diff" views where we're seeing the following error when checking the changes to a text file using a "colored diff":

An Exception Has Occurred
Python Traceback
TypeError: a bytes-like object is required, not 'str'

A "unidiff" is also broken but the error is different (the reverse?):

An Exception Has Occurred
Python Traceback
TypeError: must be str, not bytes

Diff views in "context diff" and "side by side" don't throw an exception but are much harder to parse by a person, of course.

Pygments is installed:

# pip3 show Pygments | grep Version
Version: 2.2.0
#

We don't know if this is a known issue with ViewVC 1.3.0 or if we have a missing dependency.

Edit 1: below are the system details as requested by @futatuki. CentOS 7.9.2009, Apache 2.4.6, Python 3.6.8, Pygments 2.2.0, GNU diffutils 3.3, Subversion 1.14.1

[general]
root_parents = /var/svn/repos: svn
[utilities]
diff = /usr/bin/diff
[options]
allowed_views = annotate, diff, markup, roots, tar
authorizer = svnauthz
custom_log_formatting =
  bug ([0-9]+) : https://its.site.tld/browse/BGZ-\1,
  ([A-Z]+\-[0-9]+) : https://its.site.tld/browse/\1,
  \[([3-7]{1}[0-9]{3})\] : https://its.site.tld/browse/BGZ-\1,
  (rev|revs)\.\s*([0-9]+) : https://svn.site.tld/viewvc/repo?view=revision&revision=\2
default_file_view = markup
max_filesize_kbytes = 1024
log_sort = none
hr_intraline = 1
template_dir = templates/classic
docroot = /viewvc/static
enable_syntax_coloration = 1
tabsize = 0
log_pagesize = 75
log_pagesextra = 10
[templates]
[cvsdb]
[vhosts]
[authz-forbidden]
[authz-forbiddenre]
[authz-svnauthz]
authzfile = /var/svn/etc/svnaccessfile

In diff view, "Full Colored Diff" depends on the feature of GNU diff that it treat minus value of context lines as max context lines. However, GNU diffutils removed this feature as a bug in the commit 8d26b1403e8607811ccebdfe2822f2dad42a36d3 between 3.7 release and 3.8 release. diff -U-1 foo bar is now invalid since diff 3.8.

Moreover, as we didn't check exit code and stderr output from diff command in vclib._diff_fp, in case diff exits with error without output to stdout, we treat it as "- No changes -".

So we can't get "Full Colored Diff" correctly.

When running ViewVC 1.3.0-dev under WSGI, I'm unable to navigate to a file with a non-ascii name from the directory listing that presents it. Attempting to click on a file name such as 测试文件.txt results in an exception about being unable to view 测试文件.txt. In ViewVC 1.2, this works as expected.

I'm not sure yet if this is limited to the WSGI deployment model alone, or it it also manifests under CGI. And I can't currently compare against ViewVC 1.2.x or 1.1.x, because my mod_wsgi has already been upgraded to a version requiring Python 3.x.

I had a lot of problems using code which had comments in my native language (encoded in ISO-8859-1), because UTF-8 was always chosen as default encoding. I have changed the code a bit and added more configuration switches to allow the administrator to have more influence in the encoding that is being used.

I have also found a spot where the config value detect_encoding was not queries correctly. Also, another bug was that the encoding-search-algorithm was only available in the plain text files, not in the pygmentized code.

Here is a patch I have written. Can somebody please create a PullRequest?

diff -U 2 -r viewvc_original/conf/viewvc.conf.dist viewvc/conf/viewvc.conf.dist
--- viewvc_original/conf/viewvc.conf.dist	2018-11-01 11:06:14.824994030 +0100
+++ viewvc/conf/viewvc.conf.dist	2018-11-02 18:10:17.305992089 +0100
@@ -349,4 +349,15 @@
 [options]
 
+## If the algorithm has problems detecting the encoding of a file, try the system locale (if available)
+encoding_try_locale = 1
+
+## If the detection of the encoding failed, choose this encoding as last resort
+encoding_default = iso-8859-1
+
 ## root_as_url_component: Interpret the first path component in the URL
 ## after the script location as the root to use.  This is an

diff -U 2 -r viewvc_original/lib/viewvc.py viewvc/lib/viewvc.py
--- viewvc_original/lib/viewvc.py	2018-11-01 11:06:14.836993859 +0100
+++ viewvc/lib/viewvc.py	2018-11-01 23:27:40.601850575 +0100
@@ -39,4 +39,6 @@
 import urllib
 
+import locale
+
 # These modules come from our library (the stub has set up the path)
 from common import _item, _RCSDIFF_NO_CHANGES, _RCSDIFF_IS_BINARY, _RCSDIFF_ERROR, TemplateData
@@ -1697,10 +1718,10 @@
 
 
-def detect_encoding(text_block):
+def detect_encoding(text_block, cfg):
   """Return the encoding used by TEXT_BLOCK as detected by the chardet
   Python module.  (Currently, this is used only when syntax
   highlighting is not enabled/available; otherwise, Pygments does this
   work for us.)"""

   # Does the TEXT_BLOCK start with a BOM?
   for bom, encoding in [('\xef\xbb\xbf', 'utf-8'),
@@ -1714,21 +1735,30 @@
 
   # If no recognized BOM, see if chardet can help us.
-  try:
-    import chardet
+  if cfg.options.detect_encoding:
+    try:
+      import chardet
 
-    # If chardet can confidently claimed a match, we'll use its
-    # findings.  (And if that match is 'ascii' -- which is a subset of
-    # utf-8 -- we'll just call it 'utf-8' and score a zero transform.)
-    resp = chardet.detect(text_block)
-    if resp.get('confidence') == 1.0:
-      encoding = resp.get('encoding')
-      if encoding is "ascii":
-        encoding = "utf-8"
-      return encoding
-  except:
-    pass
+      # If chardet can confidently claimed a match, we'll use its
+      # findings.  (And if that match is 'ascii' -- which is a subset of
+      # utf-8 -- we'll just call it 'utf-8' and score a zero transform.)
+      resp = chardet.detect(text_block)
+      if resp.get('confidence') == 1.0:
+        encoding = resp.get('encoding')
+        if encoding is "ascii":
+          encoding = cfg.options.encoding_default
+        return encoding
+    except:
+      pass
+
+  # We try the system's default encoding
+  if cfg.options.encoding_try_locale:
+    tmp = locale.getdefaultlocale()[1]
+    if tmp != None:
+      return tmp
 
-  # By default ... we have no idea.
-  return None
+  return cfg.options.encoding_default
   
 def transcode_text(text, encoding=None):
@@ -1750,4 +1780,19 @@
     return []
 
+  # If allowed by configuration, try to detect the source encoding
+  # for this file.  We'll assemble a block of data from the file
+  # contents to do so... 1024 bytes should be enough.
+  if not encoding:
+    if cfg.options.detect_encoding:
+      block_size = 0
+      text_block = ''
+      for i in range(len(file_lines)):
+        text_block = text_block + file_lines[i]
+        if len(text_block) >= 1024:
+          break
+      encoding = detect_encoding(text_block, cfg)
+    else:
+      encoding = cfg.options.encoding_default
+
   # Determine if we should (and can) use Pygments to highlight our
   # output.  Reasons not to include a) being told not to by the
@@ -1763,12 +1808,13 @@
                                 get_lexer_for_filename, \
                                 guess_lexer
-    if not encoding:
-      encoding = 'guess'
-      if cfg.options.detect_encoding:
-        try:
-          import chardet
-          encoding = 'chardet'
-        except (SyntaxError, ImportError):
-          pass
 
     # First, see if there's a Pygments lexer associated with MIME_TYPE.
@@ -1807,16 +1853,4 @@
   if not pygments_lexer:
 
-    # If allowed by configuration, try to detect the source encoding
-    # for this file.  We'll assemble a block of data from the file
-    # contents to do so... 1024 bytes should be enough.
-    if not encoding and cfg.options.detect_encoding:
-      block_size = 0
-      text_block = ''
-      for i in range(len(file_lines)):
-        text_block = text_block + file_lines[i]
-        if len(text_block) >= 1024:
-          break
-      encoding = detect_encoding(text_block)
-
     # Built output data comprised of marked-up and possibly-transcoded
     # source text lines wrapped in (possibly dummy) vclib.Annotation

ViewVC 1.3.0 Python Error when folders are present in CVS repository: TypeError: '>' not supported between instances of 'NoneType' and 'NoneType'

Seems to be related to the function cmp(). def cmp(a, b): return (a > b) - (a < b)

As a workaround I changed the function to: def cmp(a, b): if a == None: a = 0 if b == None: b = 0 return (a > b) - (a < b)

When changing the function, the CVS repository can be displayed even with folders present.

Describe the bug When selecting diff to the previous revision, the page that comes up looks like the binary dump of an executable.

Steps to reproduce the behavior

1. Any file in the repository
2. Click on Diff to previous X.XX

Expected behavior Before we migrated to AWS Linux, it came up with an actual diff.

Additional context Default AWS Linux install from epel:

$ yum list installed | grep -i view viewvc.noarch 1.1.26-1.el7 @epel viewvc-httpd-fcgi.noarch 1.1.26-1.el7 @epel

I went through the INSTALL file from viewvc-1.2.1, and the package seems to have done everything properly.

Here is my viewvc.conf file:

[general]
root_parents = /opt/cvs/ofc : cvs
address = Warren Lavallee <[email protected]>
[utilities]
rcs_dir = /bin/
svn = /bin/svn
diff = /bin/diff
cvsgraph = /bin/cvsgraph
[options]
root_as_url_component = 1
allowed_views = annotate, co, diff, markup, roots
binary_mime_types = application/octet-stream, image/*, application/pdf, application/vnd*, application/msword, audio/*
authorizer = forbidden
hide_cvsroot = 1
svn = /bin/svn
allow_compress = 1
template_dir = /usr/share/viewvc/templates
docroot = /viewvc-static
enable_syntax_coloration = 1
tabsize = 3
use_cvsgraph = 1
cvsgraph = /bin/cvsgraph
[templates]
diff = /bin/diff
[cvsdb]
[vhosts]
[authz-forbidden]
[authz-forbiddenre]
forbidden = cvslocks
[authz-svnauthz]
authzfile =
[query]
forbiddenre = 

There are no errors in the apache log file:

ssl_access_log:192.168.0.5 - warren [18/May/2020:08:39:10 -0400] "GET /viewvc/ofc/R1/wlthmgmt/common/feeds/xxx/inFeed_commands.sh?view=log HTTP/1.1" 200 22151
ssl_access_log:192.168.0.5 - warren [18/May/2020:08:39:12 -0400] "GET /viewvc/ofc/R1/wlthmgmt/common/feeds/xxx/inFeed_commands.sh?r1=1.10&r2=1.11 HTTP/1.1" 200 147943

I don't speak Python, but it looks to me like instead of launching a diff, it's actually including the contents of the diff binary?

Please help me out.

Versions installed:
$ yum list installed | egrep -i 'rcs|diff|svc|pygmen|cvsgra|httpd'
cvsgraph.x86_64                     1.6.1-9.el7                      @epel
diffutils.x86_64                    3.3-5.amzn2                      installed
generic-logos-httpd.noarch          18.0.0-4.amzn2                   @amzn2-core
httpd.x86_64                        2.4.41-1.amzn2.0.1               @amzn2-core
httpd-devel.x86_64                  2.4.41-1.amzn2.0.1               @amzn2-core
httpd-filesystem.noarch             2.4.41-1.amzn2.0.1               @amzn2-core
httpd-manual.noarch                 2.4.41-1.amzn2.0.1               @amzn2-core
httpd-tools.x86_64                  2.4.41-1.amzn2.0.1               @amzn2-core
perl-Algorithm-Diff.noarch          1.1902-17.amzn2                  @amzn2-core
perl-Text-Diff.noarch               1.41-5.amzn2                     @amzn2-core
python-pygments.noarch              1.4-10.amzn2                     @amzn2-core
rcs.x86_64                          5.9.0-5.amzn2.0.2                @amzn2-core
viewvc-httpd-fcgi.noarch            1.1.26-1.el7                     @epel
$ python --version
Python 2.7.16

This is the apache config file that the epel package installed:

ScriptAlias /viewvc /usr/lib/python2.7/site-packages/viewvc/bin/wsgi/viewvc.fcgi
Alias /viewvc-static /usr/share/viewvc/templates/docroot

<Directory /usr/lib/python2.7/site-packages/viewvc/bin/wsgi>
    Options -Indexes

    <Files viewvc.fcgi>
        <IfModule mod_authz_core.c>
            # Apache 2.4
            Require local
        </IfModule>
        <IfModule !mod_authz_core.c>
            # Apache 2.2
            Order allow,deny
            Allow from 127.0.0.1
            Allow from ::1
        </IfModule>
    </Files>
</Directory>
<Directory /usr/share/viewvc/templates/docroot>
    Options -Indexes
    <IfModule mod_authz_core.c>
        # Apache 2.4
        Require local
    </IfModule>
    <IfModule !mod_authz_core.c>
        # Apache 2.2
        Order allow,deny
        Allow from 127.0.0.1
        Allow from ::1
    </IfModule>
</Directory>

Since assuming the maintenance responsibility for ViewVC, I've tried to make the minimal number of disruptive changes to the various version lineages. And so we find ourselves with that latest blessed release (from the 1.1.x branch) still declaring that they'll work fine for CVS-only deployments with a Python as old as 1.5.2, and for those serving up Subversion repositories, Python 2.0. These versions of Python are ridiculously old. Like, 20 years old.

And then yesterday I came across this plea: https://snarky.ca/stop-using-python-2-6/

With that post in my head, plus the version requirements of some of the recent pull requests landing here lately, I found myself asking: How disruptive would it be to bump the 1.1.x Python requirements to say, Python 2.7 (which itself drops out of support next year in 2020)? Or at least Python 2.4, which is "only" almost 15 years old but has a good number of things that I've come to expect from Python.

I think the entirety of lib/compat.py would go away, we could use string functions instead of literally using string.foo(some_string, ...), we wouldn't to call out specific Python requirements for CVS or Subversion, etc. And more importantly, we could actually come closer to testing that the minimal requirements we specify still make sense. (I think even my oldest machines have Python 2.6 or 2.7 as their base Python version.)

Are there any operating systems left in the 90th percentile of use that don't ship with or at least provide an option for Python 2.7?

On the first request to viewvc-standalone the response shows an error in web browser. Further reloads seem to be normal. I have not yet setup any cvs ord svn repositories btw.

An Exception Has Occurred

An illegal parameter name was provided.
HTTP Response Status

400 Bad Request

The request http header browser firefox sent (taken from firefox devtools):

Host: 10.0.0.31:49152
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:67.0) Gecko/20100101 Firefox/67.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1

The response http header the browser received (taken from firefox devtools):

HTTP/1.0 400 Bad Request
Server: BaseHTTP/0.3 Python/2.7.16
Date: Wed, 12 Jun 2019 15:25:23 GMT
Content-type: text/html

I just tried with a debian 10 buster vagrant image in vagrant+virtualbox machine

Here some info to help reproduce:

[email protected]:~# apt install viewvc cvsgraph
Reading package lists... Done
Building dependency tree       
Reading state information... Done
cvsgraph is already the newest version (1.7.0-5).
viewvc is already the newest version (1.1.26-1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
[email protected]:~# head -8 /vagrant/Vagrantfile
Vagrant.configure("2") do |config|
 config.vm.box = "debian/buster64"
 config.vm.network "private_network", ip: "10.0.0.31"

 config.vm.provision "shell", inline: <<-SHELL
  #apt-get update
  #apt-get upgrade
  apt-get install -y nano git autoconf apache2 php php-mysql perl gettext mariadb-server make imagemagick
[email protected]:~# viewvc-standalone --host=10.0.0.31
server ready at http://10.0.0.31:49152/viewvc
10.0.0.1 - - [12/Jun/2019 15:17:07] "GET /viewvc HTTP/1.1" 400 -
10.0.0.1 - - [12/Jun/2019 15:17:07] ViewVC exited ok
10.0.0.1 - - [12/Jun/2019 15:17:28] "GET /viewvc HTTP/1.1" 200 -
10.0.0.1 - - [12/Jun/2019 15:17:28] ViewVC exited ok
10.0.0.1 - - [12/Jun/2019 15:17:28] "GET /viewvc/*docroot*/styles.css HTTP/1.1" 304 -
10.0.0.1 - - [12/Jun/2019 15:17:28] ViewVC exited ok
10.0.0.1 - - [12/Jun/2019 15:17:28] "GET /viewvc/*docroot*/images/viewvc-logo.png HTTP/1.1" 304 -
10.0.0.1 - - [12/Jun/2019 15:17:28] ViewVC exited ok

Describe the bug

The filename is not properly escaped, and HTML in a filename is rendered in the browser. This is when viewing the changed file in a revision, behind the "View Changeset" link.

Steps to reproduce the behavior

1. Create a SVN repository with a file in it named like this:

<h1 onmouseover=alert`XSS`>XSS

2. Visit http://localhost:49152/viewvc/Development?revision=1&view=revision

The path shows XSS in big letters, and when the mouse moves over it a JavaScript popup shows.

Expected behavior Path shows <h1 onmouseover=alert'XSS'>XSS.

Additional context

This is in the current master version, cf136ba4e3cf9.

Recently upgraded to Debian 11 and noted that there was no viewvc package available https://tracker.debian.org/pkg/viewvc Debian seems to think that viewvc is no longer supported. I installed the current master branch along with the python3-subversion package and can report that all seems good on Debian 11. If there are any issues then I haven't found them yet. It would be good if viewvc 1.3 could be made the basis of a resurrected viewvc for Debian.

• conf/viewvc.conf.dist, lib/config.py (enable_debuglines): New configuration option.
• lib/viewvc.py (Request.add_debugline): New. Sprinkle a few invocations of this throughout.
• templates/classic/_footer.ezt, templates/default/_footer.ezt: Add logic to use JavaScript to log the debuglines to the console.
• docs/template-authoring-guide.html: Document the new template variables.

For some reason I do not get ViewVC working on Ubuntu 22.04.1 LTS with Subversion.

The webpage just gives this message: An Exception Has Occurred Python Traceback

I check as advised under python:

# python3
Python 3.10.6 (main, Nov  2 2022, 18:53:38) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from svn.core import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'svn.core'
>>> "%s.%s.%s" % (SVN_VER_MAJOR, SVN_VER_MINOR, SVN_VER_PATCH)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'SVN_VER_MAJOR' is not defined

I also tried to check the svn modules in python:

>>> help('modules svn')

Here is a list of modules whose name or summary contains 'svn'.
If there are any, enter a module name to get more help.

svn
svn.admin - svnadmin calls for managing a repository.
svn.common
svn.common_base
svn.config
svn.constants
svn.exception
svn.local
svn.remote
svn.utility
libsvn
libsvn._client
libsvn._core
libsvn._delta
libsvn._diff
libsvn._fs
libsvn._ra
libsvn._repos
libsvn._wc
libsvn.client
libsvn.core
libsvn.delta
libsvn.diff
libsvn.fs
libsvn.ra
libsvn.repos
libsvn.wc
pysvn
pysvn._pysvn
pysvn._pysvn_3_10

I check which packages are install: subversion: Installed: 1.14.1-3ubuntu0.22.04.1 python3-subversion: Installed: 1.14.1-3ubuntu0.22.04.1 python3: Installed: 3.10.6-1~22.04

Hmmmm ... after reading all above I tried this in python:

>>> from libsvn.core import *
>>> "%s.%s.%s" % (SVN_VER_MAJOR, SVN_VER_MINOR, SVN_VER_PATCH)
'1.14.1'

Does this mean on Ubuntu the naming is different svn.core vs libsvn.core? Not sure how to solve this but any help would be appreciated.

Thanks, Matthijs

As of 1.3.0, the oldest supported Subversion implementation requires version 1.14.0 of Subversion. But the codebase still has compatibility logic dating back to much, much older Subversion versions. Let's clean that up.

Not sure if this is a feature request or a documentation issue, but Google hasn't found any results that says this is already supported.

Is there (or could there be) an option in the ViewVC log to show the equivalent of the "-g" / "--use-merge-history" option on the CLI SVN tool?

This unrolls the changes from an SVN merge, and shows the log entries for every merged change.

People used to using this option on the CLI tool tend not to include the full text of every merged change in the "merge commit", because it is available with this option, but when using ViewVC to browse the repository it means it is hard to get more details on the changes from the merges