Integrate[Integrate[20000*E^y/(1-x/2),{x,-5,0}],{y,-3,0}]+Integrate[Integrate [20000*E^y/(1+x/2),{x,0,5}],{y,-3,0}] is much slower in Mathics than maxima.

./admin-tools/make-op-tables.sh builds the JSON tables.

Also, environment variable MATHICS_CHARACTER_ENCODING can be used to set $SystemCharacterEncoding and the initial value of $CharcterEncoding.

Some adjustments to tests with DifferentialD were made so we use standard Unicode symbols not WMA unicode.

There was a bug in gstest from a prior commit in mathics-scanner where the "pre-scanner()" was replacing strings starting with "|".

I had a hard time understanding the flow of what's going on, and this process was both painful and made me a bit annoyed.

I was pleased though at @mmatera start to move the makeboxes into its own module, and we have a little bit of segregating boxing routines, so that I suppose helps a little.

It is not that the code isn't sophisticated or complicated. It is just that it is not organized and changes haven't been coordinated.

Let me describe a little of the history around this PR and the code I see, and reflect. The aim show of this reflection shows how the code has become of poorer quality, is in constant need of serious refactoring, and can get worse with uncoordinated bug fixes or feature improvements. Functionally the code does a quite a bit; but it is at the cost of lots of code with undue complexity. I assume that much of the complicated code is to ensure correct behavior.

However, where we find that the code is not correct or lacking, we have a hard time figuring out where to go and what to change that isn't going to impact a lot of other things.

mmatera notes a problem with infix operator behavior in the ascii-op-to-Unicode branch. I had sort of seen this when I wrote that branch initially. One thing that I noticed that was was wrong in my PR is that the place where the change occurs felt wrong: it is code inside a "MakeBox" rule. But this rule is defined on class creation or registering each of the infix operators. These rules are is never changed. Something like $CharacterEncoding can change at will, so this location is too static.

Therefore I had to give up (temporarily) on making this work for $CharacterEncoding. Instead, I used the less dynamic $SystemCharacterEncoding instead.

Looking over this in a second pass, I see now that a rule like this is boneheaded:

formatted_output = 'MakeBoxes[Infix[{%s}," %s ",%d,%s], form]' % ( replace_items, operator, self.precedence, self.grouping, ) default_rules = {

...

"MakeBoxes[{0}, form:InputForm|OutputForm]".format(op_pattern): formatted_output } Do you see why? ...

Hint: it is in the " %s " part.

This is supposed to be a rule that is called when boxing infix operators. And already it is deciding that operators should be surrounded by spaces for "InputForm" and "OutputForm". In other words, right here in part which dictates rules to Box Infix expressions, we are already making decisions about the low-level formatting: that there should be spaces around the operator and what characters to use to represent the operator.

And when we actually get to the low-level format to final string, we have lost structure. Basically we have violated the principle that you evaluate to get an M-expression, then you Box the result, and then after Boxing then a low-level formatting is done.

If we want to write dozens more forms, the above isn't scalable. And we make high-quality formatting harder.

At some level this was probably noticed, at least implicitly; when there are such few comments it is hard to know what was noticed and what was just random programming by reacting to problems that come up. Since everything can't be done by MakeBox rules, we have do_format_xxx routines in mathics-core/mathics/core/formatter.py as well as special routines in builtin/arithfns/basic.py, builtin/pympler/asizeof.py, some that are done on inside MakeBox() internal functions.

In short, since principles if they were decided or defined, they were not communicated anywhere I can tell; so naturally formatting codes is scattering at many of places in the code at several conceptual levels. Possibly some of the code is redundant or worse, works cross purposes.

If discussion of high-level principle were lacking, so are basic description code. Things like docstrings on functions. Or making an effort to name what a function does. For example take the get_op() an interanal function inside evaluation method apply_infix() leaving aside the missing apply_ prefix that I have mentioned many times before.

What is get_op() "getting"? You already pass it some sort of operator. If you look at the function you realize it is not getting or accessing something, but rather converiting or formatting.

And then once you realize that you are then in a position to ask why is this routine formatting inside a Boxing routine? In the overall architecture isn't a separate step. Or should low-level formatting routines, be put together?

Vagueness in code, lack of description and discussion around the code has led to very haphazard code that, in the end result, does not seem fully thought out.

With all the effort spent in just figuring out what he code does; by the time I understand that, I am exhausted and often not in a mood to think about how it should be designed or whether it follows a design or how to best write this.

Let me come back to adding spaces around the operator and losing the operator structure (the " %s " part) one more time. Because of this, the "remedy" was put in place that makes things worse. A routine was added that scans strings and unconditionally changes some strings (e.g. ASCII-formatted operators) into Unicode characters. I doesn't matter if this was done by ignorance, willful frustration: in the end, it makes the code a bigger mess.

It is, as I say, like trying to solve a Rubik's cube by getting adjacent faces in line one at a time. In the beginning there is a certain satisfaction because you can "make progress". However getting to the end this way is much harder than understanding what is going on performing operations that work in conjunction with the principles and groups of the Rubik's cube.

Given all this, my inclination right now is to hold off on adding new Forms, or correcting the existing ones to be more correct. Instead if we just make things work they do but in a logical sensible and extensible way, I think this would buy us the most to then correct the existing behavior to match the specifications more closely and to add more Forms.

@rocky, maybe you can help me with this. For some reason, all the test pass if I enable the line that singletonizes the Symbol class, except for one test in combinatorica. As you know better that code, could you help me investigating why this happens?

If this starts to work, I guess that we can get an improvement in performance if we use the id() of the symbols instead of the string name the symbol in the pattern matching routines.

I tried to implement Diagonal and it seems working. In https://github.com/Mathics3/mathics-core/issues/115 is proposed to use native Wolfram Language, but I hope this it will be sufficient, at least for the moment.

A number of different kinds of atoms that can't change value unnecessarily go through an extra expansion for fixpoint check.

In particular many constants values: strings, numbers, fixed symbols e.g. True, False, N, etc.

The one place where we do need to check though is Symbol, because a value like "x" may change. So we separate Symbols from Constants.

Also we remove "import" of a name where that name isn't used directly in the module. This seems to happen so that some other module can find the name instead of where it was defined. This isn't helpful. In fact, it is the opposite.

Using 4.0 on Linux, I get the following

mathics

Mathics 4.0.0
on CPython 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
using SymPy 1.10.1, mpmath 1.2.1, numpy 1.22.4

Copyright (C) 2011-2021 The Mathics Team.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions.
See the documentation for the full license.

Quit by evaluating Quit[] or by pressing CONTROL-D.

In[1]:= Integrate[(1 - x^3)^(2/3)/x,x]
Out[1]= --1 ^ (2 / 3) x ^ 2 Gamma[-2 / 3] hyper[{-2 / 3, -2 / 3}, {1 / 3}, 1 / x ^ 3] / (3 Gamma[1 / 3])

Notice the --1 output. This is an error. Mathematica says

PreDecrement::rvalue: 1 is not a variable with a value, so its value cannot be changed.

Btw, as a side issue, what is hyper[ function above? There is no such function in Mathematica.

But the main problem is generating --1

Integrate[Integrate[1,{y,0,E^x}],{x,0,Log[13]}] gives Integrate[ConditionalExpression[E, {True}], {x, 0, Log[13]}] when Integrate[Integrate[2,{y,0,E^x}],{x,0,Log[13]}] or Integrate[Integrate[0,{y,0,E^x}],{x,0,Log[13]}] or even Integrate[Integrate[0.9999999,{y,0,E^x}],{x,0,Log[13]}] works.

We get perhaps a 10%-20% improvement:

In[1]:= Timing[AbsFast[I + 3]]
Out[1]= {0.00704927, Sqrt[10]}

In[2]:= Timing[AbsFast[I + 3]]
Out[2]= {0.00116204, Sqrt[10]}

In[3]:= Timing[Abs[I + 3]]
Out[3]= {0.00120341, Sqrt[10]}

In[4]:= Timing[Abs[I]]
Out[4]= {0.00029366, 1}

In[5]:= Timing[Abs[I]]
Out[5]= {0.00030883, 1}

In[6]:= Timing[AbsFast[I]]
Out[6]= {0.000244551, 1}

Note that due to caching, in comparisons, we should run an expression twice and use the second (cached) value.

Specific specialization:

We can fold in knowledge of the specific sympy and mpmath functions (Abs and fabs) that get called rather than having to look them up.

Because there is exactly one argument, we don't need argument looping mechanisms.

In general, things like this.

This PR is a previous step for adding a Pyston test workflow (see #368). The proposed changes allow skipping the tests associated with symbols whose evaluation requires not available external libraries.

To check that indeed this works with a minimal environment, I added here a workflow to check that in standard Ubuntu Python.

In this PR, a basic proposal about how to handle List evaluations is implemented. Same ideas could be implemented to other frequently used symbols, or certain direct sympy expressions.

Here is the next step in the road to use Symbols as a singleton class. With this #PR, in my machine, I could reduce in 30s the 4min 30s ' docpipeline runtime.

Most of the evaluation process involves to compare Symbols, look for definitions and reformat Expressions accordingly. Until now, each time we rebuild an expression object, we had to recreate Symbols by their names. This triggered class constructors and string formatting checking. On the other hand, each time that a Symbol was compared against another expression, we had to look at its name and make str comparisons. With the new implementation, a big part to that overhead is eliminated. The next step would be to reformulate in the same way has_form, and finally, the Definitions object.

This is the first step in my proposal of refactoring MakeBoxes. The main change here is that MakeBoxes rules are applied in mathics.eval.makeboxes.eval_makeboxes instead of the mathics.core.expression.rewrite_apply_eval_step. In a next step, makeboxes rules are going to be stored not as downvalues of the MakeBoxes symbol, but as format values.

• [x] Import["ExampleData/MadTeaParty.gif"] is broken
• [x] Show SciPy version (or non) under version info (and in Django)
• [x] (Django) Go over gallery examples, possibly add new stuff. color boxes are too small
• [ ] Asymptote axes for Plot are misaligned possibly now that they are aligned for SVG
• [ ] LaTeX doc rendering of Asymptote graphs is misaligned. Go over LaTeX doc for content
• [x] Remove Combinatorica error in trying to redefine CatalanNumber
• [ ] Go over text in docs for links and text.
• [ ] Replace all methods named apply which are really eval methods? And remove acceptance of apply in the contribute() function?
• [ ] Go over CHANGES.rst (which is really part of the release itself.

I would like to be able to load a Mathics notebook into TeXmacs and work with it in the native MakeBoxes format of Wolfram language notebooks instead of being forced to convert it to HTML/CSS or LaTeX first.

Will also post this request on TeXmacs mailing list as an idea for a new plugin.

Description

1. Attempting to install Mathics-omnibus gets stuck at Installing build dependencies ... |.
2. Attempting to install Mathics3 raises the following error:

gcc -pthread -DNDEBUG -O2 -fPIC -I/home/hyperswine/.pyenv/versions/pypy3.9-7.3.10/include/pypy3.9 -c lib/recordclass/_litelist.c -o build/temp.linux-x86_64-3.9/lib/recordclass/_litelist.o
lib/recordclass/_litelist.c: In function ‘litelist_resize’:
lib/recordclass/_litelist.c:31:56: error: ‘PyLiteListObject’ {aka ‘PyListObject’} has no member named ‘ob_item

How to Reproduce

1. From a popos (ubuntu 22.04) environment with pypy 3.9.15, run pip install Mathics-omnibus. And get the error in 1
2. From the same environment, run pip install Mathics3. And get the error in 2

Expected behavior

Not get stuck at installing build dependencies and not raise an error. I did check the recordclass source code, and it seems like the PyLiteListObject struct did not in fact contain ob_item. Unless Im seeing things, that does seem like a problem with that dependency, so I actually don't know why Mathics3 works when its including that dep.

Your Environment

• PopOS 22.04, Intel + Nvidia
• Pypy 3.9.15 through pyenv
• No other mathematica /mathics installs on the system

@mmatera and @TiagoCavalcante : this is kind of important to understand.

Although we now require a frontend to explicitly call initialize_system() and this may seem more awkward, it is helpful and important for a couple of reasons.

First: it provides better modularization and reduces circular imports.

mathics.builtin is close to top-level import and, in contrast to mathics.core.system_init there are a lot of modules underneath mathics.builtin. (There are none under mathics.core.system_init).

In general, it is considered bad Python coding practice to put a lot of code in __init__.py files.

As a result, we can have more import mathics.core.system_init from a module where we can't do the same for import mathics.builtin.

Second: by placing initialization of structures which do not change outside of development we are in a better position to write an administrative tool to load all of this and then dump it as a system image file, which can be read in rather than built it up from scratch as happens now. This new function initialize_system might have a parameter to read the image file rather than do the work it does now.

In its current form though initialize_system() does as not encapsulate system loading as much as it could.