Socks and Sandals

I was thinking about AppEngine some more today and it occurred to me that not only could AppEngine be responsible for a lot of people learning/using Python, but it very well might be Python’s answer to Rails. Its so-called “killer app”, if you will.

Up until this point, Python has been plagued with multiple, competing Web frameworks all taking some mindshare and there really hasn’t been a strong rallying point in the Python Web community like Rails. It appears that Django has been winning out in the blogosphere lately, but its nothing like Rails’ devout following in Ruby-land.

However, with AppEngine, Google does Rails one better: instead of just making it easy to code your app, they make it just as easy as Rails to code and dirt simple to deploy and reduce the operation maintenance to near zero. The need for things like ActiveRecord and migrations is pretty reduced in the AppEngine environment, as is all but a tiny knowledge of SQL (called GQL in that realm). That’s really, really attractive if you’re a Web consultancy shop that’s looking to turn over clients as fast as possible or a side project with a mandate for speed, quality and low cost. To me, that seems like it would be worth learning a little Python for.

AppEngine is pretty clearly aimed at Facebook’s F8 platform but it could end up hitting Python with a major boost in popularity as an aside. I bet Guido is smiling all the way to the bank on this one…

I just read a little bit about the recent announcement of Google AppEngine and I think its a pretty good service overall for certain kinds of Web applications. Unlike the rumors that were floating about prior to its announcement, its not a competitor to AWS directly, nor even a competitor to Ning as some have also claimed. To me, it seems more directly in competition with Heroku if such a thing could even be said. It is clearly based on Bigtable, though, so that part of the rumor appears to have been true.

Being so simple has some advantages and presents some interesting constraints. Because you don’t have root and can’t run the “box” yourself, you’re forced to think simply about the app itself. This seems as if it would be quite a welcome constraint for a lot of Web developers who don’t care about running a network. But the constraint really serves to enable Google-style scalability at its heart. As well, the use of CGI allows Google to run your code wherever they deem it best at the moment underneath, without you having to care about that sort of thing. This is something they already do quite well.

As well, the lack of a traditional RDBMS is something that obviously works well in a shared-nothing environment and makes a lot of sense since Google’s infrastructure is already based on such. This gives real credence to the idea that the RDBMS isn’t the be-all-end-all and will introduce a lot of developers out there to this new way of thinking.

A few things I don’t like:

• Users of an AppEngine app must login with Google Accounts
• You do have to upload your Python source to Google and this leads to obvious privacy and IP concerns
• No recurring job scheduling (essential for lots of different kinds of apps)

One more obvious thing is that an application built on AppEngine is one that is much, much easier for Google to acquire than one that is not. Don’t underestimate that piece of it, as its likely this service will be a loss-leader for Google.

Finally, I think this announcement will be very good for Python. As AppEngine only supports Python right now and for the foreseeable future, anyone interested in AppEngine will have to learn some Python. This will shed some more light on Python for people who might not have otherwise given it a try.

UPDATE: Apparently, others agree about the acquisition potential of apps on AppEngine. Good stuff.

...is automatic syntax checking. It really sucks to run a Python script for 15 – 30 minutes only to find that you forgot to import a package you need.

Perl and Ruby have the -c option built into the interpreter to check syntax, but both do this as a matter of course before execution in order to catch issues of this nature. Python only has PyChecker which is pretty good, but not as well integrated as Perl or Ruby’s syntax checking.

Just something for other Perl/Ruby-to-Python switchers to keep an eye on, that’s all.

I’ve just released by code to update EC2 instances on start up and shut down with the Nettica DNS service. You can find it here:

http://code.google.com/p/netticadns/

With this code, you can set up your EC2 instances to automatically register/deregister themselves with the Nettica DNS service on start up and shut down. There’s no RPM packaging at this point, but there is a dpkg available for it.

I’d appreciate any feedback or patches. Have fun with it :)

My last post somehow managed to create quite a stir on Reddit and news.YC. This was not my intent, since I was pretty sure no one actually read my blog and this is the first of my articles to make it to any news distribution site. I’d like to take this opportunity to respond to some of the comments and also share some of my experiences with the first week of Python.

First of all, thanks to all the people with encouraging comments. I really appreciate all of the support as this was not a decision I made lightly. I have a lot of experience with Ruby and had a lot invested in it so moving to Python was a big thing for me.

I’d also like to respond to some of the recurring themes in the comments:

• As for my use of “stone-clad”: obviously, that makes no sense unless you’re talking about dinosaur fossils. Thanks for pointing that out. Doh!
• This somehow became a bone of contention on the Reddit comment forum: Ruby’s blocks are, in fact, a form of coroutine. Please refer to this page on coroutines if you need some help with their definition.
• As for my like of user-level threads over native threads: This is a complicated issue, but I’d rather have the option of both than just one. Using a lot of cheap cooperative threads over a few native threads yields big scalability and performance and softens the blow of concurrent programming. If I had to choose one, I’d choose cooperative threads and just spawn processes to account for multiple cores. Check out how Erlang does it to see why I prefer this mode.
• As for Python’s whitespace: I tend to like it. It makes Python slightly more annoying to write, but I think it pays bigger dividends when it comes to readability. However, I may just be blowing smoke here since I’ve only been using it for a week ;)
• Some Perl people came to its defense in the comments. I don’t begrudge you or anyone else their choice of language and I know some really great programmers that love Perl (big up, MCT). Perl has a solid runtime, its very flexible and has a ton of good documentation and libraries that make it a real contender for almost any task. To them, I’d say that I find it to be less than sublime to program in but I respect what can be done with if you’re willing to put in the time to really come to grips with the language.
• Apparently, there are now tools to debug memory issues in Ruby, most notably Rublique. Thanks for pointing those out.
• A filesystem can indeed be written in lots of languages that you might not think of since it will be I/O-bound most of the time. This makes raw speed of execution less important. Particularly in the case of a system like GFS where you have disk and network I/O that are sucking up the vast majority of wall-clock time. It may never be as fast as GFS or Hadoop, but it can work quite well regardless. I plan to redo that effort in either Stackless or Erlang at some point in the future.

And some specific responses:

• Fitzgerald Steele: You had me rolling on the floor laughing :) I wasn’t sure anyone would get it :-P
• Matt S Trout: The Perl2exe that I was referring to was this one which doesn’t appear to come from ActiveState. However, your point is well taken.
• n d: I never even got to the point of using rubyscript2exe. My tests exposed the issue on Windows before I even had the chance to try to package it into a Windows executable.
• winston: I’m pretty hurt. I pumped Rubinius in my post and I didn’t submit this to any news site, nor did I expect anyone but a few friends to read it. As for your main point, I would prefer to use the Ruby language for most of my stuff, but the runtime beat up on me too many times.
• David: By your rational, we should all be using Algol or Fortran since they have really deep roots. Somehow, I don’t see that happening.
• The guy on Reddit who said the title was a “like a textual goatse”: That was even funnier than Fitzgerald Steele’s comment!

I’ve been using Python for a little over a week now professionally and I’ve come to find out more about it. So far, its going pretty well. Some points:

• The ZSI interface is not as nice as SOAPpy when you just want to get something going. SOAPpy is really, really nice. Plus, the current version of ZSI (2.0) doesn’t support SOAP headers which makes it pretty useless for real-world SOAP to a big, public API right now. If you have to work with SOAP, stick to Python <= 2.4.x as long as you can so you can keep using SOAPpy.
• Python’s object attributes seem to be somewhat lamer than Ruby’s. Using property() and creating getter and setter methods is less handy-dandy than Ruby’s attr_* helpers. This may not be required, but I don’t know any better right now.
• The reference material says that the .pyc and .pyo objects created by a higher version should be importable by a lower version, but I’ve found this not to be the case.

That’s about it for insights so far. I’m really liking it. I thought I’d miss some beats switching to Python but so far its been more than sanguine.

This past week, I switched from programming in Ruby to programming in Python. Why did I do that? Well, that’s a story with a bunch of background.

Coming to bat

I started playing with Ruby in 1999-2000 at Villanova for small stuff. I had read that seminal Dr. Dobb’s article by the PragProg guys. I bought the first edition of the Pickaxe book and then had to buy it again after I destroyed it from reading/referencing it too much. At one point there, I was running the first FTP mirror for the Ruby sources in the Western hemisphere. I also met David Black way back in 2001(2?) at a Ruby Tuesday’s (pun intended) in Trenton back when he was still a Communications professor at Seton Hall and I was still a college student. That was the only meeting of the “East Coast Ruby Users Group” ever held.

I thought Ruby had a lot of really cool concepts in it. (built-in coroutines, true OO, closures, heavy-duty metaprogramming, etc.) I also had little experience with Perl so I didn’t appreciate how similar they were. Villanova was a Java school and Ruby was like a real breath of fresh air compared to the Machiavellian verbosity imposed by that language. I was really just struck with the compactness of the language and how much you could pack into a very short amount of code.

Strike one

After school, I got an internship then real job with a local startup working in the now-hot email reputation/authority space. My boss there was also a Ruby fan and recognized its ability to really pound out some serious stuff in a very short amount of time. This is something you require in a startup situation.

We were using Ruby as a systems administration glue language at first here and there, but when we started building the SpamSquelcher prototype, we decided to use Ruby for its Web interface. We chose what was basically the only option for a Web framework at the time in Ruby, a compact MVC piece called Roach. Now that I go searching for the URL for Roach, I can see that it hasn’t moved even a patch ahead of where it was since we started using it. Funny. I think maybe IOWA was out at the time, too.

Roach was supposed to be MVC, but the “M” part was lacking as it had no integrated database functionality. However, for our purposes (controlling an embedded appliance), this did not matter at all so we had no issue with this limitation. Development went quickly, but there was a nagging issue with a memory leak. Every hit to the app server would irrevocably raise its memory usage and nothing we tried ever made a big difference. We could see no problem with the Ruby code itself, neither ours nor Roach’s, but there were no tools for diagnosing memory problems in Ruby apps. There still aren’t any.

We eventually had to just ship it with a hard memory limit under daemontools so that it would never chew up too much RAM on an appliance where RAM was already at a premium. So, when we became TurnTide the interface was rewritten in PHP and one of the reasons for it was this issue. I never did figure out what the problem was, either.

Strike two

So, later, a while after Symantec had purchased TurnTide, I was looking at ways to process our very large amount of data from the Brightmail and TurnTide streams. While we didn’t require anywhere near the truly massive powers that Google needs, I was (and still am) enamored by their internal architecture and sought to replicate that for our use. I didn’t find out about Nutch and Hadoop until much later. So, I basically rebuilt all of the functionality of Google File System and their Map/Reduce framework in Ruby in about 5 weeks.

However, it never actually worked. The tests passed and the functionality was good, in terms of correct Ruby code, but the “chunkserver” nodes would very rapidly just start using all the RAM on the box and freeze up when the filesystem became even moderately loaded. I was not able to determine the cause of this issue at the time and the project was cancelled. I came to find out later what its cause was but it was too late. This remains my only professional failure to ship to date.

Some light in this tunnel

It wasn’t all bad. Most of the Ruby I was writing was working out fine. It was small scripts to do this or that and the productivity was great. Its especially awesome for text processing because its just as handy as Perl but more readable when you want to reuse a script six months later.

Also, I wrote a small Rails app to manage our PXE installations of appliances in a day that saved us all any more trips to the server room to reinstall a box. That was a big boost. Rails is pretty great framework and I wasn’t even using it for the kind of thing it was designed for. (no RDBMS in my app)

Strike three; you’re out

Segue to this past week. I was writing a internal cross-platform utility to do some HTTPS communication and it had to run on multiple platforms, including Windows. Most of our people run Windows, so supporting this platform is a stone-clad requirement. I had finished the utility in a pretty quick amount of time and was planning on packaging it up for Windows with rubyscript2exe. However, when I started testing on Windows, I found that the program would just hang when it got to the final HTTPS request.

Turns out that the One-Click Installer version of Ruby on Windows has a known (!!) problem with combinations of Ruby threads and I/O. Given that the net/protocol.rb library that underlies all of Ruby’s standard Internet protocol functionality wraps most calls in a Timeout::timeout() call and that call is implemented with a Ruby thread, the problem seemed pretty large. But, I found a post denoting the problem and indicating that Ruby built under MinGW did not exhibit this issue.

I think, if you’ve read this far, that you can probably tell that it, in fact, does exhibit this issue. Both the binary package I downloaded (ruby 1.8.4) and the one I built from source under MinGW/MSYS (ruby 1.8.6). I posted to ruby-talk about it here, citing the hope-rendering link from ruby-talk about MinGW.

Enter the Python

At this point, I was looking at having to do some serious Ruby internals hacking on a platform that I know little to nothing about or rewriting the utility in a different language. I was T minus 2 days to deadline. I chose the latter and was able to rewrite and finish the whole thing in Python in a day and a half without ever having touched Python before.

Obviously, most of the speed of this rewrite was due to the fact that I was merely porting it to Python from an existing Ruby implementation. I am not making any claims of Python having greater productivity than Ruby in the development cycle. I am claiming two things, though:

1. Python was very productive, despite having never used it before, and
2. The end result actually fucking worked

That last one is kind of big with me.

Python? Really?

I had given some thought to doing the rewrite in Perl but two things stopped me from going that route. I really did not want to have to support any Perl code over any length of time as it really (to me) is a butt-ugly language and is mnemonically taxing. Also, the perl2exe compiler to produce Windows executables costs money. Python has the py2exe package that did the job very nicely and is free. C was obviously out due to time constraints even though I know that language better than any other.

Plus, Python seemed a more natural fit for a port from Ruby as I knew it contained most of the high-level constructs I had come to love in Ruby. There was only one function that I had to truly rewrite due to Python’s lack of “blocks”, or one-way coroutines.

I had heard at some point in the past that Guido was planning on getting rid of the map(), filter() and reduce() functions in Python 3000 and always wondered what he would want to do that for. Now I get it: Python’s list comprehensions can perform all of those jobs in a natural syntax.

The regular expression support is not as natural as Ruby’s or Perl’s but its not significantly more verbose or hard to work with.

My only beef was the lack of YAML support in the Python standard library, but I was using YAML in Ruby for a configuration file and Python’s ConfigParser turned out to work just fine in its stead.

One thing Python has all over Ruby is documentation. I used the Python docs from python.org almost exclusively and that was good enough to get the job done. Their documentation is first-class and this is an area where Ruby knows it needs to catch up.

And, there are a bunch of things available to a Python guy that Ruby just can’t compete with that are of particular interest to me. Two that come to mind immediately are Twisted and Stackless Python. The former was used by others at TurnTide for creating a really powerful SMTP testing tool and the latter was used by TurnTide’s competitor IronPort to build one of the industry’s best MTAs.

Along this same vien, I had previously tried creating a binding for Linux’s epoll(4) facility for Ruby but was stymied by Ruby’s internal workings. A far better programmer than I also fell victim to this same issue.

The Fundamental Issue

In case you somehow missed it, the fundamental issue above in all three of the problems was the Ruby runtime. I have no qualms with the language and I think its got tons of great things going for it. However, the runtime sucks. I’ve said this privately for some time but now I’m going on record.

The community seems to realize this and is moving in a number of different directions, most notably the YARV runtime. However, I fundamentally disagree with this direction. YARV will remove two of the awesome features of the language, continuations and green threads, and it will keep the same GC model. This makes no sense to me, as the GC is clearly one of the weakest parts of the current runtime. As well, Erlang and Stackless draw their powers from NOT supporting native threads as part of the language and they are far better suited to the future of heavy concurrency than Ruby. Rubinius is far more interesting to me, but they have a much longer climb up the hill towards usefulness.

What about the Twitter phenomenon?

Some might say that I should have stepped up and fixed the issues I was having and not just complain about them on my blog. To them, I’d say that in the first two cases I tried and in the third I didn’t have the time. I’m not taking a “wait and hope” attitude like some others, hoping that someone else will fix my problems. I’m circumventing them altogether. Above all other things professional, I have a job to do and employers to report to.

And furthermore, for me, programming languages and their runtimes are tools. I think they are for a lot of us. You wouldn’t continue to use a hammer whose head keeps flying off when you take a swift backswing, would you?

So, no more Ruby? Really?

I will probably still fall back on Ruby for text-processing scripts as its always been stellar for those kinds of tasks. But, I have to say that any real software I write will no longer be in Ruby for the foreseeable future. I am discounting Rails here: I think Rails is great for what it is and I’ve had nothing but good experiences with it precisely because it is so “opinionated”. Its a very productive environment and manages to successfully avoid the runtime problems that I’ve personally hit in the past. I hope that the Ruby community can really bang out a worthwhile runtime in the coming year and if they do, I’m back on board. If I get some time, I might even help out where I can.

For now, though, I am looking forward to using more and more Python and getting to know that language, as well as some Erlang in there somewhere, too. I have the beta of the new Armstrong book and am tearing through it with the same rapaciousness as I did the first edition of the Pickaxe.