Thursday, October 1, 2009

Duplication of effort or evolution?

Dear LazyWeb,

I need help compiling a list of Python modules that exist outside of the standard library and either duplicate or serve as replacements for modules that are in the standard library.

For example, simplejson became the standard library module json but is also maintained separately for backwards compatibility and to include the optional C extension that is not part of the standard library version (please correct me if I'm wrong).

Another example is httplib2, which is meant to have more features than the standard library's httplib.

If you know of other similar modules, please post links in the comments. Thanks!

22 comments:

Paul Moore said...

lxml has an ElementTree compatible API, I believe, so lxml as a whole is effectively a superset of xml.etree.

Vinay Sajip said...

There's argparse, which could replace getopt and optparse, and which is being considered for inclusion in the stdlib (PEP 389).

vinces1979 said...

mx.Datetime which offers much of the same features of datetime but also has some nice added functionality

John Gabriele said...

Doug, would be great to see a Python wiki page made from the list.

Doug Hellmann said...

@Paul - Thanks for the tip about lxml, I haven't used that and wouldn't have known about it otherwise.

@Vinay - I'll include argparse in my "evolution" list.

@vinces1979 - Good one!

@John - We'll see what sort of list I come up with when I'm done. I have a couple of ideas for how to use it, but posting it online is certainly an option.

Thanks everyone, keep them coming!

Ian Bicking said...

One might say nose and py.test extend the anemic built-in unittest runner.

wsgiref has an WSGI HTTP server, but there are quite a few other servers (CherryPy, mod_wsgi, paste.httpserver, flup, Spawning, Tornado, and a couple more I can't remember).

INITools is a (generally compatible) replacement for ConfigParser. Of course there are many other config parsers available, for INI or INI-like syntax.

lxml and BeautifulSoup are replacements for HTMLParser.

Ian Bicking said...

Oh, some more: Twisted is a replacement for asyncore. There are a couple other similar async setups too.

There's a curl library which replaces urllib/urllib2/httplib

Werkzeug includes a HTTP POST body parser, replacing cgi.FieldStorage

Weberror is an alternative to cgitb

Setuptools and Distribute are kind of alternatives to distutils.

The path module replaces parts of os.path and glob.

py.lib has a replacement for logging. Twisted has its own logging too (not sure if it is notable).

Sphinx replaces pydoc.

There's a couple pattern matchers replacing re, can't remember their names though.

I just saw something on PyPI that replaces robotparser.

There's a bunch of things that could be considered to replace shelve: ZODB, Durus, and a few other object databases.

xturtle replaces turtle.

Doug Hellmann said...

Thanks, Ian, those are all great suggestions! I should crowd source all of my column ideas. :-)

Pythonic Avocado said...

configobj which is a replacement for configparser

John Paulett said...

Related to simplejson is cjson and demjson.

xml.ElementTree development is handled outside the stdlib.

There is a 2.4/2.5 backport of multiprocessing / fork of pyprocessing

John Paulett said...

Not sure if this counts, but Django includes backports of some code:

functools vs django.utils.functional
json vs django.utils.simplejson

Georg said...

Doug: Python's standard json *does* include a C extension.

Doug Hellmann said...

@Georg - Ah, my mistake, thanks!

cj_ said...

One could argue BeautifulSoup is a replacement for HTMLParser.

Lucio Torre said...

pyparsing for parsing stuff

configglue is a library that glues together python's optparse.OptionParser and ConfigParser.ConfigParser, so that you don't have to repeat yourself when you want to export the same options to a configuration file and a commandline interface.

alex dante said...

opterator is a very nice wrapping around optparse.

shove is a (highly underrated IMO) replacement for shelve.

Maniac said...

I dare to suggest that html5lib will be even better than BeautifulSoup.

Krys Lawrence said...

Hi Bob,

decorator and some part of DecoratorTools could be argued to replace functools.wraps (and related).

Krys Lawrence said...

Oops. I meant "Hi Doug". I had Bob Ippolito on the brain by mistake. :)

Sorry.

Anonymous said...

For the sake of completeness, do you want to consider packages like SQLAlchemy and epydoc.

Doug Hellmann said...

I'm not sure what one could say SQLAlchemy duplicates in the stdlib, per se. It extends sqlite by sitting on top of it, but I'm really looking for things that are meant to substitute. shove, mentioned in an earlier comment, is an excellent example.

Anonymous said...

dateutil is very handy. In particular, it handles parsing fractional seconds unlike strptime in the stdlib.