New challenges

Life has been pretty busy recently. I worked on my Google Summer of Code project implementing lattices in Sage; I continued working on several optimization projects, mostly dealing with the charging of multiple electric vehicles (in cooperation with FH Joanneum Kapfenberg and other companies); I took my last exams and finished my master’s thesis (“Optimization of a Purlin Punching Process”). I’m glad (and a little proud) that all of this worked out so well.

Adding to that, I organized my move to the U.S., where I finally landed and already began working for Wolfram Research. I am working as a Kernel Developer on Mathematica Online (or whatever it will be called once it gets released to the public). This is quite related to my previous work on Mathics.

Now, the sad news is that I have to stop actively coding on Mathics as a consequence. However, I see it in really good hands with Angus having already taken the lead recently, and he will certainly continue to keep Mathics flourish. Others are jumping in as well, so Mathics will not disappear, but grow further. Of course, I will still be there when assistance is needed, to merge in pull requests, and to keep the server running.

I’m really excited about the new challenges. I’ll share some of my experience on a special (rather personal) blog Go West, Young Jan. It’s in German so I don’t forget my mother’s tongue completely.

Let the adventure begin.

Mathics: Django in Sage-Python on nginx and gunicorn

I recently moved Mathics from my “general” web server that is also running tripedia.org, fairteiler.com, this site, and lots of other stuff, to its own virtual machine (a KVM). Mathics uses a lot of special stuff (e.g., not the regular Python, but Sage) and occasionally tends to freeze the whole machine, so it makes sense to separate it from the rest.

In the light of things to come (thinking of “live” collaboration and long polling) I thought Mathics should “go green” with the potential to serve it asynchronously. The Python WSGI HTTP server gunicorn seems like an obvious choice to serve Django applications this way, and it plays together nicely with the HTTP proxy server nginx. Everything runs on Debian Linux.

In addition, I installed Sage 5.0 (this is as simple as extracting the files to an arbitrary location, running make and waiting for a couple of hours). Because Sage’s version of Python (sage -python) will be used to run Django and gmail is used via SSL to send emails (registration/password and Django’s error emails, in particular), Sage has to be rebuilt with SSL support:

apt-get install libssl-dev
sage -f python

nginx is configured by adding a file to /etc/nginx/site-enabled containing something like this:

  upstream app_server {
    server unix:/tmp/gunicorn.sock fail_timeout=0;
  }

  server {
    listen 80 default;
    client_max_body_size 4G;
    server_name _;
    keepalive_timeout 5;

    location / {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect off;
      proxy_buffering off;
      proxy_pass http://app_server;
      break;
    }

    location /media/ {
      root /home/jan/static;
    }
  }

This will serve everything under /media/ from /home/jan/static (a symbolic link pointing to mathics/web/media), while all other URLs are passed to the app server which communicates with gunicorn via Unix sockets.

One way to deploy gunicorn is Circus, a process watcher and runner. It can be configured like the following:

[watcher:mathics]
cmd = sudo -u jan /home/jan/sage/sage -python /home/jan/sage/local/bin/gunicorn_django --bind unix:/tmp/gunicorn.sock --workers 4 /home/jan/mathics
working_dir = /home/jan/mathics
send_hup = true

This will execute gunicorn_django in the Mathics directory and listen to the corresponding Unix socket at startup. gunicorn will in turn fire up four workers.

I couldn’t figure out how to conveniently restart these Django worker processes. However, after simply killing them using

kill `ps -ef | grep "python /home/jan/sage/local/bin/gunicorn_django" | grep -v grep | awk '{print $2}'`

they will be restarted automatically, thereby loading any changes made to the Mathics source code.

Please let me know if you have any suggestions.

Mathics goes open source

I’ve been working a lot on Mathics again during the last weeks. All towards the goal that I’ve had in mind for a long time: to release it as open source. After all, that’s the only thing that could set Mathics apart from Mathematica—the only thing that’s not so good about Mathematica is that it’s not free, neither as in freedom nor as in free beer.

If you wonder what Mathics is: it’s a general-purpose computer algebra system implementing the awesome Mathematica mathematics/programming language. Some of its most important features are

  • a powerful functional programming language,
  • a system driven by pattern matching and rules application,
  • rationals, complex numbers, and arbitrary-precision arithmetic,
  • lots of list and structure manipulation routines,
  • an interactive graphical user interface right in the Web browser using MathML (apart from a command line interface),
  • creation of graphics (e.g. plots) and display in the browser using SVG,
  • an online version at www.mathics.net for instant access,
  • export of results to LaTeX (using Asymptote for graphics),
  • a very easy way of defining new functions in Python,
  • an integrated documentation and testing system.

Read the manual to learn more about the over 350 built-in functions and symbols in Mathics.

The actual heavy math stuff (like integration) is mostly done by the Python package SymPy. There is optional support for functions depending on Sage as well. Despite “out-sourcing” most mathematical functions, Mathics has more than 20,000 lines of Python code already, dealing with much non-trivial stuff such as parsing Mathematica input, pattern matching, graphics generation, etc.

Unfortunately, Firefox is the only browser supported so far, because no other browser supports MathML yet. However, this is expected to change pretty soon when Webkit (used by Safari and Chrome) adds MathML support. I wonder whether Internet Explorer will ever get that far.

I set up a project homepage at www.mathics.org for organizational stuff, while www.mathics.net is still the place for the online interface of Mathics. The source code is hosted at github.

I hope that one day there will be developers joining me. Contact me if you want to get involved!