unbump.
October 30, 2009 11:11 AM Subscribe
SAGE is a free, open-source computer algebra system.
SAGE is notable for bringing together a number of existing (and extremely fast) open source projects under a single umbrella, and religiously avoiding redundant code.
The idea is to create a free alternative to closed-source systems like Mathematica, Maple, and MATLab. In spite of its buffet-style approach to computation, it already runs considerably faster than Mathematica in many situations.
Importantly, open-source software is easier to ethically cite in academic papers, since the code is available for review. As a result, a number of researchers (including me) regularly use SAGE in their work, using its massive computational power to find new results to prove.
SAGE is notable for bringing together a number of existing (and extremely fast) open source projects under a single umbrella, and religiously avoiding redundant code.
The idea is to create a free alternative to closed-source systems like Mathematica, Maple, and MATLab. In spite of its buffet-style approach to computation, it already runs considerably faster than Mathematica in many situations.
Importantly, open-source software is easier to ethically cite in academic papers, since the code is available for review. As a result, a number of researchers (including me) regularly use SAGE in their work, using its massive computational power to find new results to prove.
This looks interesting. I don't have too much experience with MATLAB, but Mathematica and Maple are pretty buggy and crash-prone, to the extent that I'm genuinely shocked they are still in business.
posted by Blazecock Pileon at 11:23 AM on October 30, 2009
posted by Blazecock Pileon at 11:23 AM on October 30, 2009
There's also GNU Octave which is an open source matlab clone.
posted by delmoi at 11:27 AM on October 30, 2009 [1 favorite]
This looks interesting. I don't have too much experience with MATLAB, but Mathematica and Maple are pretty buggy and crash-prone, to the extent that I'm genuinely shocked they are still in business.I've done a little bit with an old version of matlab (6.0) and I didn't have any trouble with it. Are you using the Mac versions of Mathematica and Maple?
posted by delmoi at 11:27 AM on October 30, 2009 [1 favorite]
Yeah, Octave is pretty good. And free.
posted by Blazecock Pileon at 11:28 AM on October 30, 2009
posted by Blazecock Pileon at 11:28 AM on October 30, 2009
I administered a Mac-based Mathematica & MATLAB computer lab for a couple of years. For me the biggest problem with Mathematica was the licensing system. What an unbearable headache in the institutional environment. You have two choices: either manually enter unique license keys for every machine every so many months, or run your own licensing server and maintain the keys from there. The server sounds appealing until you realize that the software is incredibly braindead: you can DoS attack the server by simply opening up a telnet connection on that port and leaving it open. The server would happily refuse all other connections forever.
So, anyway, hooray for competition, especially ones that don't have licensing headaches. Hopefully Wolfram will be spurred not only to improve the software proper but also the licensing scheme.
posted by jedicus at 11:41 AM on October 30, 2009
So, anyway, hooray for competition, especially ones that don't have licensing headaches. Hopefully Wolfram will be spurred not only to improve the software proper but also the licensing scheme.
posted by jedicus at 11:41 AM on October 30, 2009
One can hope. The licensing for Matlab is also a complete nightmare. However, if you are performing simulations and doing control design, such as with Simulink, there aren't many options. Matlab is an absolutely fantastic, very powerful program, and the toolkits for Simulink are really great. I looked into using Scicos but it was sorely lacking in the needed functionality.
posted by molecicco at 11:49 AM on October 30, 2009
posted by molecicco at 11:49 AM on October 30, 2009
Yeah, the Mathematica licensing is an incredible pain. All of the closed-source alternatives are expensive and a bit of a pain to maintain. As a result, almost no one uses them outside of the computer lab setting, which really keeps people from using these systems as learning tools.
If you give smart people tools like this that they can play with in their free time, then the chances of getting cool results outside of the academic mainstream greatly increases. Professional math-heads under the publish-or-perish regime tend to stick to familiar areas where they know they can get funding and results. Giving massive computational power to the masses along with a bit of time for play allows people to try things out free of institutional pressure, and maybe see things that really draw them into the subject, or achieve understanding outside of a class context.
posted by kaibutsu at 11:57 AM on October 30, 2009
If you give smart people tools like this that they can play with in their free time, then the chances of getting cool results outside of the academic mainstream greatly increases. Professional math-heads under the publish-or-perish regime tend to stick to familiar areas where they know they can get funding and results. Giving massive computational power to the masses along with a bit of time for play allows people to try things out free of institutional pressure, and maybe see things that really draw them into the subject, or achieve understanding outside of a class context.
posted by kaibutsu at 11:57 AM on October 30, 2009
The licensing for Matlab is also a complete nightmare.
Is it? We were using a pre-OS X version that had to run under Classic, and its licensing was fairly straightforward. I guess the licensing has gotten worse since then, despite competition from Octave, which is kind of disheartening.
posted by jedicus at 11:58 AM on October 30, 2009
Is it? We were using a pre-OS X version that had to run under Classic, and its licensing was fairly straightforward. I guess the licensing has gotten worse since then, despite competition from Octave, which is kind of disheartening.
posted by jedicus at 11:58 AM on October 30, 2009
And for free, open-source statistical computing (and publication-quality plots), there's R (or "GNU S"), an extensible environment based on S from Bell Labs. Back in the day, they talked about S for "exploratory data analysis."
posted by Hello Dad, I'm in Jail at 12:16 PM on October 30, 2009 [2 favorites]
posted by Hello Dad, I'm in Jail at 12:16 PM on October 30, 2009 [2 favorites]
Importantly, open-source software is easier to ethically cite in academic papers, since the code is available for review.
What?
posted by squorch at 12:26 PM on October 30, 2009
What?
posted by squorch at 12:26 PM on October 30, 2009
Depends on the licensing structure you are under. I was at a university where our division only had a limited license, and we had to go through a complicated procedure to purchase the license for any specific toolkits we wanted. And then they were only usable on one computer in our network at a time. Although, if I closed down Matlab and a colleague wanted to give Model Predictive Control a whirl, we often had to wait until the next day for the license to free up (even after I rebooted). Admittedly, half of the complication was from the's school's own administrative structure.
posted by molecicco at 12:41 PM on October 30, 2009
posted by molecicco at 12:41 PM on October 30, 2009
'Importantly, open-source software is easier to ethically cite in academic papers, since the code is available for review.'
What?
The argument goes something like this: If I find that something is statistically significant using a closed-source program, how can you be sure it's actually significant and not the result of an error in the program? For some things it's easy to check, but for others it can be time-consuming to check the result in another program, preferably an open source one.
Where the argument breaks down, of course, is that if I cite results found using an open source program, it may be theoretically possible for someone to formally verify that the program produced the correct result in that instance, but it's highly doubtful that any given reader ever will or indeed that anyone ever will. There is not much evidence that open source development results in better or more correct programs and there may never be; there are just too many confounding variables: popularity, project maturity, language used, development style (agile, waterfall, etc), involvement of paid programmers, project size, etc as well as too few good apples to apples comparisons like Matlab vs Octave.
Of course, just because there isn't any proof that open source development tends to lead to better programs doesn't mean there aren't or can't be good open source programs or even ones that are better than their closed source competitors. It just means that statements about the ethics of citing a result from a closed vs open source program are interesting in theory but not in practice.
posted by jedicus at 12:44 PM on October 30, 2009 [3 favorites]
What?
The argument goes something like this: If I find that something is statistically significant using a closed-source program, how can you be sure it's actually significant and not the result of an error in the program? For some things it's easy to check, but for others it can be time-consuming to check the result in another program, preferably an open source one.
Where the argument breaks down, of course, is that if I cite results found using an open source program, it may be theoretically possible for someone to formally verify that the program produced the correct result in that instance, but it's highly doubtful that any given reader ever will or indeed that anyone ever will. There is not much evidence that open source development results in better or more correct programs and there may never be; there are just too many confounding variables: popularity, project maturity, language used, development style (agile, waterfall, etc), involvement of paid programmers, project size, etc as well as too few good apples to apples comparisons like Matlab vs Octave.
Of course, just because there isn't any proof that open source development tends to lead to better programs doesn't mean there aren't or can't be good open source programs or even ones that are better than their closed source competitors. It just means that statements about the ethics of citing a result from a closed vs open source program are interesting in theory but not in practice.
posted by jedicus at 12:44 PM on October 30, 2009 [3 favorites]
Cool, I saw this mentioned by Paul Lutus (yeah, that Paul Lutus) on a Reddit IAmA a few days ago when he was asked what he was currently working on. Apparently he is an active contributor.
posted by sophist at 12:49 PM on October 30, 2009 [1 favorite]
posted by sophist at 12:49 PM on October 30, 2009 [1 favorite]
How far down do you need to go to verify it? The open source criticism does always apply fully to things like MATLAB, as many researchers use custom packages or open source programs built on top of existing closed source platforms for the algorithms that are relevant to the research. For instance my lab uses SPM for much of our fMRI data analysis, which is a GNU project built on top of MATLAB. You can go in and look at the routines in SPM and verify those, but not all of the underlying MATLAB routines they are calling. By the same token however, if you are running it on Windows or Mac then they are eventually going to be calling computational routines in the kernal (even if they are trivial) that you cannot go in and verify either.
posted by sophist at 12:56 PM on October 30, 2009 [3 favorites]
posted by sophist at 12:56 PM on October 30, 2009 [3 favorites]
I forgot a really important link in the post:
sagenb is a good sandbox where you can try out Sage without downloading and installing.
A couple pointers for trying stuff out:
1) Tab-completion rocks. So if you type 'mat' and then hit Tab, you'll get a list of stuff that might be what you're looking for. (In this case, math, matlab, matrix, mathematica, matlab_console, matrix_plot, etc. I was thinking of 'matrix', but I guess there's a some matlab and mathematica interoperability available if you want it.)
2) To get help and examples on any given command, you can type the command followed by a question mark:
matrix?
gives the documentation on the matrix function and a bunch of examples.
To see the source executed by any given command, you type the command with two question marks:
matrix??
good luck!
posted by kaibutsu at 1:08 PM on October 30, 2009 [1 favorite]
sagenb is a good sandbox where you can try out Sage without downloading and installing.
A couple pointers for trying stuff out:
1) Tab-completion rocks. So if you type 'mat' and then hit Tab, you'll get a list of stuff that might be what you're looking for. (In this case, math, matlab, matrix, mathematica, matlab_console, matrix_plot, etc. I was thinking of 'matrix', but I guess there's a some matlab and mathematica interoperability available if you want it.)
2) To get help and examples on any given command, you can type the command followed by a question mark:
matrix?
gives the documentation on the matrix function and a bunch of examples.
To see the source executed by any given command, you type the command with two question marks:
matrix??
good luck!
posted by kaibutsu at 1:08 PM on October 30, 2009 [1 favorite]
You can use sage as a web browser interface as well as rpy support for R. (According to wikipedia, at least).
---------------------------------
According to
$ sudo apt-get -s install sagemath
it's in the ubuntu package repositories. I need to go find 827MB of space, now.
---------------------------------
(On preview, this point has been addressed:)
'Importantly, open-source software is easier to ethically cite in academic papers, since the code is available for review.'
What?
It means you can present a result, and the source code you used to get there, and other people can replicate the work without:
1) Buying Maple, Mathematica, or Matlab
and/or
2) Looking damn carefully at the source code for Maple, Mathematica, or Matlab
to make sure it's not lying/broken.
It's not just "Did the Mathematica notebook work out right?" but ALSO is the Mathematica kernel right?
posted by sebastienbailard at 1:25 PM on October 30, 2009
---------------------------------
According to
$ sudo apt-get -s install sagemath
it's in the ubuntu package repositories. I need to go find 827MB of space, now.
---------------------------------
(On preview, this point has been addressed:)
'Importantly, open-source software is easier to ethically cite in academic papers, since the code is available for review.'
What?
It means you can present a result, and the source code you used to get there, and other people can replicate the work without:
1) Buying Maple, Mathematica, or Matlab
and/or
2) Looking damn carefully at the source code for Maple, Mathematica, or Matlab
to make sure it's not lying/broken.
It's not just "Did the Mathematica notebook work out right?" but ALSO is the Mathematica kernel right?
posted by sebastienbailard at 1:25 PM on October 30, 2009
I've been using Matlab for ten years, and still do some stuff with it at work. As a programming language, Matlab used to be really horrible. For example, its for-loop was so incredibly inefficient that "optimizing" code meant writing matrix algebra operations only and doing several orders of magnitude more actual computation which, lacking a JIT compiler, was faster. Unfortunately, at the time there was nothing better available and Matlab got quite popular, despite its bad programming language design. The language has gotten better over the years and now even the for-loop is quite efficient, but it suffers badly from being grown and not actually designed. Python is far better and covers the same features.
My background is in computer vision, where Matlab is still pretty much the standard for everything non-realtime. For academic institutions, it is quite cheap (as in beer) and mostly the incentive for using Octave, NumPy, Sage and/or R instead has been ideological. But as the open source alternatives are actually becoming faster and better than Matlab, things could change quite soon.
posted by ikalliom at 1:34 PM on October 30, 2009
My background is in computer vision, where Matlab is still pretty much the standard for everything non-realtime. For academic institutions, it is quite cheap (as in beer) and mostly the incentive for using Octave, NumPy, Sage and/or R instead has been ideological. But as the open source alternatives are actually becoming faster and better than Matlab, things could change quite soon.
posted by ikalliom at 1:34 PM on October 30, 2009
Well, if you do your research in matlab, you should be able to run your code in GNU Octave, if you're worried about the unverifiability of the closed-source matlab.
posted by delmoi at 1:35 PM on October 30, 2009
posted by delmoi at 1:35 PM on October 30, 2009
As a person too stupid to do math beyond prerequisite calculus courses, I will post the one thing related to Matlab that I know of.
Sorry
posted by mccarty.tim at 1:51 PM on October 30, 2009
Sorry
posted by mccarty.tim at 1:51 PM on October 30, 2009
SAGE is a free, open-source computer algebra system.
SAGE is a moderately inexpensive, open-ethics computer system administrator group.
Sage is a freely growing, open growing season food flavoring system.
SAGE SAGE Sage SAGE SAGE SAGE SAGE
Sage...pro?
posted by Ogre Lawless at 1:52 PM on October 30, 2009 [1 favorite]
SAGE is a moderately inexpensive, open-ethics computer system administrator group.
Sage is a freely growing, open growing season food flavoring system.
SAGE SAGE Sage SAGE SAGE SAGE SAGE
Sage...pro?
posted by Ogre Lawless at 1:52 PM on October 30, 2009 [1 favorite]
Most of the time, research code written in some version of Matlab works just barely after some fidgeting in your version Matlab. Running it in Octave will require some work, after which you'll discover that plotting the results does not work or that the code uses a single function from a toolbox which does not exist in Octave. It is all very demotivating, although not difficult as such.
mccarty.tim, that was hilarious!
disclaimer: I'm a member of the facebook group "I ♥ MATLAB"
posted by ikalliom at 2:01 PM on October 30, 2009
mccarty.tim, that was hilarious!
disclaimer: I'm a member of the facebook group "I ♥ MATLAB"
posted by ikalliom at 2:01 PM on October 30, 2009
Crap, I guess I should mouse-over the post, carry on...
posted by The Power Nap at 2:22 PM on October 30, 2009
posted by The Power Nap at 2:22 PM on October 30, 2009
As a bonus, after installing SAGE, you also get a fully functional installation of Python (including all the cool and often hard to install packages such as SciPy, PIL, MatPlotLib, etc). Just write
./sage -python
in the command line.
In my experience, installing SAGE can be the easiest method to get SciPy and friends working on some configurations.
posted by lenny70 at 2:44 PM on October 30, 2009 [1 favorite]
./sage -python
in the command line.
In my experience, installing SAGE can be the easiest method to get SciPy and friends working on some configurations.
posted by lenny70 at 2:44 PM on October 30, 2009 [1 favorite]
Where the argument breaks down, of course, is that if I cite results found using an open source program, it may be theoretically possible for someone to formally verify that the program produced the correct result in that instance, but it's highly doubtful that any given reader ever will or indeed that anyone ever will.
However doubtful it is, it's still infinitely more probable than anyone inspecting and verifying a closed source program, which is impossible.
posted by DU at 5:10 PM on October 30, 2009
However doubtful it is, it's still infinitely more probable than anyone inspecting and verifying a closed source program, which is impossible.
posted by DU at 5:10 PM on October 30, 2009
Where the argument breaks down, of course, is that if I cite results found using an open source program, it may be theoretically possible for someone to formally verify that the program produced the correct result in that instance, but it's highly doubtful that any given reader ever will or indeed that anyone ever will.
I think it reflects a general attitude of mathematicians; they just don't want proofs with a chunk marked [trust mathematica on this bit] or [for this step, you and your students will have to buy software]. It would be like presenting a physicist with test equipment that he or she isn't allowed to analyze, test, modify, or reverse-engineer. It would start to get to you after a while.
By taking this stance, mathematicians are shaping what kind of world they want to live in.
posted by sebastienbailard at 7:06 PM on October 30, 2009
I think it reflects a general attitude of mathematicians; they just don't want proofs with a chunk marked [trust mathematica on this bit] or [for this step, you and your students will have to buy software]. It would be like presenting a physicist with test equipment that he or she isn't allowed to analyze, test, modify, or reverse-engineer. It would start to get to you after a while.
By taking this stance, mathematicians are shaping what kind of world they want to live in.
posted by sebastienbailard at 7:06 PM on October 30, 2009
they just don't want proofs with a chunk marked [trust mathematica on this bit] or [for this step, you and your students will have to buy software].
And people shouldn't have to depend on proprietary software for the kind of basic mathematical computations these packages provide. I say it's a right, a right of all people, to be able to do basic calculations without paying somebody else to do them.
posted by twoleftfeet at 9:35 PM on October 30, 2009
And people shouldn't have to depend on proprietary software for the kind of basic mathematical computations these packages provide. I say it's a right, a right of all people, to be able to do basic calculations without paying somebody else to do them.
posted by twoleftfeet at 9:35 PM on October 30, 2009
jedicus: "Where the argument breaks down, of course, is that if I cite results found using an open source program, it may be theoretically possible for someone to formally verify that the program produced the correct result in that instance, but it's highly doubtful that any given reader ever will or indeed that anyone ever will."
If I might give that argument a bit more life, I think there's an important thing being overlooked here. Source code is the preferred input for doing automated formal and informal verification. You can already run source code through tools like splint for simple error analysis, though formal verification is still something of an open problem. It seems to me that SAGE would be an ambitious and interesting target for automated verification. I think computing power is growing faster than academic publication, so it's not a forgone conclusion that all software used in pursuit academic publication might be verified. Just maybe not within your lifetime.
The other challenge is that academic publishing today isn't compatible with software. If you write a paper and write your own software to support it, your journal offers no support in publishing the software you wrote; the end result is poorly archived source code that disappears quickly. I've seen some downright atrocious Perl code emitted by biologists in support of a finding; you can't even download it anymore. In this case, I think the ACM and IEEE need to show a good faith effort and lead the way; far too many of our own papers focus more on the output of an unpublished toy program intended to demonstrate some algorithm than the algorithm itself.
posted by pwnguin at 10:14 PM on October 30, 2009 [1 favorite]
If I might give that argument a bit more life, I think there's an important thing being overlooked here. Source code is the preferred input for doing automated formal and informal verification. You can already run source code through tools like splint for simple error analysis, though formal verification is still something of an open problem. It seems to me that SAGE would be an ambitious and interesting target for automated verification. I think computing power is growing faster than academic publication, so it's not a forgone conclusion that all software used in pursuit academic publication might be verified. Just maybe not within your lifetime.
The other challenge is that academic publishing today isn't compatible with software. If you write a paper and write your own software to support it, your journal offers no support in publishing the software you wrote; the end result is poorly archived source code that disappears quickly. I've seen some downright atrocious Perl code emitted by biologists in support of a finding; you can't even download it anymore. In this case, I think the ACM and IEEE need to show a good faith effort and lead the way; far too many of our own papers focus more on the output of an unpublished toy program intended to demonstrate some algorithm than the algorithm itself.
posted by pwnguin at 10:14 PM on October 30, 2009 [1 favorite]
Unbump? No entiendo.
'Sage' sort of means 'make this go away' on image bbs-es like 2chan or 4chan (NSF?).
Because normal comments 'bump' the thread and keep it alive, while 'sage' has no effect or the opposite effect.
posted by sebastienbailard at 9:57 AM on October 31, 2009
'Sage' sort of means 'make this go away' on image bbs-es like 2chan or 4chan (NSF?).
Because normal comments 'bump' the thread and keep it alive, while 'sage' has no effect or the opposite effect.
posted by sebastienbailard at 9:57 AM on October 31, 2009
On the bright side, though, it's easy to publish code in preprints on the Arxiv, which is quickly becoming the goto-place for math papers. Dead-tree journals don't have the space to publish code, but the arxiv has effectively infinite space. So I can say, 'Here's the code I used to generate my base cases, and here's the code that checked that the theorem is true in those cases. My induction argument takes over from there.'
Another strong argument for opening the source code is that a broad base of research mathematicians can often just find better ways of doing things than a handful of corporate coders tasked with writing tools for the researchers. We really do look at the code (matrix??), and occasionally see a better and/or faster way of performing a calculation.
An unnamed researcher I know did a bunch of work with an unnamed closed-source computer algebra system, developing a free extension of that system. In that capacity, they worked with the closed source of the original system, and by all accounts made a huge number of improvements. Keeping the source a secret keeps such improvements from happening.
posted by kaibutsu at 12:23 PM on October 31, 2009
Another strong argument for opening the source code is that a broad base of research mathematicians can often just find better ways of doing things than a handful of corporate coders tasked with writing tools for the researchers. We really do look at the code (matrix??), and occasionally see a better and/or faster way of performing a calculation.
An unnamed researcher I know did a bunch of work with an unnamed closed-source computer algebra system, developing a free extension of that system. In that capacity, they worked with the closed source of the original system, and by all accounts made a huge number of improvements. Keeping the source a secret keeps such improvements from happening.
posted by kaibutsu at 12:23 PM on October 31, 2009
I say it's a right, a right of all people, to be able to do basic calculations without paying somebody else to do them.
Well, you don't need software to do any of the calculations these programs do...
posted by abc123xyzinfinity at 8:42 PM on October 31, 2009
Well, you don't need software to do any of the calculations these programs do...
posted by abc123xyzinfinity at 8:42 PM on October 31, 2009
« Older Frankfurter Buchmesse | Gimme a ... Newer »
This thread has been archived and is closed to new comments
posted by wrok at 11:17 AM on October 30, 2009 [1 favorite]