Eric Day

Yet Another Language Comparison

• Don't be Different - Disregard all that hoopla about everyone being their own unique, beautiful butterfly - sometimes it's best to conform. If I want to hack on the Linux kernel, I'm probably going to be doing it in C or assembly. If I want to contribute a plugin to Drupal or Wordpress, it's going to be in PHP. Even though it is technically possible to embed some other language into a project, it's usually easiest to take the path of least resistance. There is only one work flow, one set of development tools, and less time spent context switching between the different languages. If there is some precedent for a language already, stop and use that. The rest of this post should be applied when you have a clean slate and can make choices without worrying about tight integration with an existing project.
  
  
• Be Mature - I recently read this article suggesting we need a new programming language, and while the ideas are nice, I can't say I agree. As great as new languages like Go may be, you never know when the plug may be pulled on the core development team. There is always the option of maintaining the language tools as well as your project, but that's a lot of extra work. I like to choose languages that I know are not going anywhere or won't be changing too drastically in the future.
  
  
• Be Popular - There are a few websites out there that try to measure programming language popularity from various sources. Take them with a grain of salt, but it does give you a pretty good idea of who is hot or not, and even what the recent trends are. If a language doesn't appear in the top 20-30 of the general lists, I usually don't look any further. Google Trends can be useful as well to create your own trending graphs from their data set. Popularity is important because you want to have useful developer tools and a community to help answer your questions. If you and a professor at some university are the only people using a language, there is going to be a bottleneck on resources. This is also critical for open source projects when you want to build a developer community. Choose a language that folks already know so they can make useful contributions and help make your software better.
  
  
• Determine What's Important - As with many aspect of computer science (and life), choosing a language is all about trade-offs. To make these decisions, you need to know what limits you are going to hit. Are you going to be bound by CPU, disk, network, user, or some other resource? By user bound I mean your application will always be limited by user interaction, and no hardware resource limits will ever be hit. If you are CPU bound, you probably want a machine code or efficient byte-code compiled language. If you are user bound, performance matters less so you have more options. Keep in mind these limits are not isolated and can have effects on one another. For example, some languages may make I/O interaction really easy at the cost of space, but this may be due to double or even triple buffering of data, causing your CPU usage to increase too.
  
  
• Don't Guess - I found myself performing a number of micro-benchmarks to test various aspects of the languages. How long does it take to call a function? How much overhead is there in the concurrency primitives? How expensive is context switching? How efficient are the built-in string processing functions? It's best to answer these questions by writing small programs in each language and comparing the results.
  
  
• Choose the Best Tool for the Job - Sometimes you choose a language mainly because it has a particular library, module, or some built in feature suitable for your application. Most languages have the same standard library bits, but many have a few niche uses and have great support for certain features. For example, if you're going to be running on large multi-core machines and need to share a lot of memory between threads (so not multi-process), languages that have a single interpretor lock like Python may not be a good choice. If you want a simple, integrated webserver framework that you can customize, Python is great. C or C++ may not be the best choice in this case because you'll mostly be writing your own. Examine what your primary feature requests are and how well they are met by each language.
  
  

My Current Preferences

• Web Applications - Early on I used Perl for all my web programming, then I switched to PHP for a number of years, and recently I've found myself preferring Python. It is a fantastic language and allows you to do almost anything with the objects at runtime (for better or worse). I've been doing some work on OpenStack and have found the WSGI standard great for building modular web applications. Combined with an event framework like Eventlet, you don't even really need Apache httpd. The main concerns with Python are CPU bound tasks and SMP support because of the global interpretor lock (GIL). Most of the web apps I write are not bound by either and most of the heavy lifting (if any) is pushed to some other service that is more efficient (like a database). Projects like Django and Pylons take this to the next level providing frameworks around these basic ideas, but if you want to keep it simple then 10 lines of Python will get you a functioning web server and WSGI application (with a dependency on Eventlet). The Routes and SQL Alchemy packages also provide some very useful functionality while building your web applications.
  
  
• Scripting, Tools, and Middleware - For these types of apps, I've mainly used Perl or a combination of shell/sed/awk, but recently I've again found Python to be a better fit. Decent versions of Python are standard on any system now, so you don't need to worry about customizing or installing any dependencies to get your applications running. Again, if there are SMP or CPU performance concerns, you might need to look at another language.
  
  
• Shared Libraries and Drivers - These consist of libraries that are used to provide some core functionality or other service. For example, libz for compression or libmysql to talk with MySQL servers. You really want the lowest common denominator so the library can easily be wrapped and reused in a number of other languages. This means writing it in C. Python, PHP, Perl, Erlang, Ruby, Lua, and pretty much all others have well defined interfaces for interacting with C libraries. Projects such as SWIG even take care of some of this interfacing work for you, allowing you to build multiple language bindings at once. You can of course write your driver in each language natively, but this can be a lot of work. You can probably get away with writing the library in C++, but you'll most likely run into more issues than if you had just used C.
  
  
• Servers - This is where most of my time has gone throughout my career, and for about 10 years the answer was always C. I was always trying to squeeze every bit of CPU and memory out the servers I was writing. In the past three years I started doing a lot more C++ work for MySQL related projects like Drizzle, and recently I've been experimenting with a number of alternatives. In a previous blog post I tested performance and throughput for a few different solutions, and I while I was impressed with the higher level languages, the C++ version still won by a good margin. In further tests I performed more CPU-intensive calculations and the Javascript and Python versions went through the roof compared to C++. This was most likely due to less time being spent in the kernel for the I/O calls, which should be about the same regardless of language. There were two languages that did stand out in the performance tests: Go and Erlang. Even with heavier CPU loads, they both performed quite well, usually taking only 10-15% more time than the C or C++ equivalents. Go is still a no-go due to it's immaturity, but I think Erlang is a real contender. I've been somewhat frustrated with C++ due to it's verbosity and nuances. For example, defining and debugging complex template code can be a nightmare, but it's required if you want to use the STL. When doing the same thing in Erlang, I found myself writing more concise code with less bugs in a fraction of the time. In other words, the code was almost as fast and much more elegant than the C or C++ equivalents.
  
  

And the winner is...

Scale Stack vs node.js vs Twisted vs Eventlet

The Test

var net = require('net');
net.createServer(function (socket) {
  socket.on("data", function (data) {
    socket.write(data);
  });
  socket.on("end", function () {
    socket.end();
  });
}, {backlog: 32768}).listen(12345, "localhost");

from twisted.internet.protocol import Protocol, Factory
from twisted.internet import epollreactor
epollreactor.install()
from twisted.internet import reactor
 
class Echo(Protocol):
    def dataReceived(self, data):
        self.transport.write(data)
 
factory = Factory()
factory.protocol = Echo
reactor.listenTCP(12345, factory, backlog=32768)
reactor.run()

import eventlet
 
def handle(fd):
  while True:
    c = fd.recv(16384)
    if not c: break
    fd.sendall(c)
 
server = eventlet.listen(('0.0.0.0', 12345), backlog=32768)
pool = eventlet.GreenPool(size=32768)
count = 0
while True:
  new_sock, address = server.accept()
  pool.spawn_n(handle, new_sock)

Setup

root             soft    nofile          65535
root             hard    nofile          65535
*                soft    nofile          65535
*                hard    nofile          65535

echo 32768 > /proc/sys/net/core/netdev_max_backlog
echo 32768 > /proc/sys/net/core/somaxconn
echo "21000 61000" > /proc/sys/net/ipv4/ip_local_port_range
echo 8192 > /proc/sys/net/core/rmem_default
echo 8192 > /proc/sys/net/core/wmem_default

Results

Graph of the result listed below.

Conclusions

Update - August 6, 2010

• Scale Stack 50.25
• node.js 117.04
• Twisted 89.46
• Eventlet 77.80
• Erlang 61.65
• Python threads 111.04 (lots of memory even with minimal stack size)
• Go v1 62.95
• Go v2 59.73

Threads with Events

So, what is the "best" paradigm?