Thursday, October 18, 2012

Python Web Frameworks PEP8 Consistency

The code is read much more often than it is written. The PEP8 guidelines are intended to improve the readability of code. Readability counts, no doubt, but readability consistency is important, it is equally important to know when to be inconsistent. The report below makes excuse for the following:
E501 line too long (> 79 characters)
E231 missing whitespace after ',:'
W291 trailing whitespace
W293 blank line contains whitespace
In this post we will take a look at web frameworks source code readability. The ratio between a web framework total python source lines to PEP8 errors found represents PEP8 error rate in respectful framework:
  1. bottle
  2. cherrypy
  3. circuits
  4. django
  5. falcon
  6. flask
  7. pyramid
  8. pysi
  9. tornado
  10. turbogears
  11. web.py
  12. web2py
  13. webapp2
  14. wheezy.web
The source code is hosted on bitbucket, let clone it into some directory and setup virtual environment (this will download source code per framework listed above).
hg clone https://bitbucket.org/akorn/helloworld
cd helloworld/04-pep8 && make env up
The make file has a target for each metric, so in order to gather pep8 count issue make pep8, or make count to count total lines.
Since some web frameworks consist of several packages (developed by the same team), I have combined them this way:
flask += jinja2 + werkzeug
pyramid += chameleon + webob
Here are raw numbers (November 24, 2013):
number of source lines:
bottle => 7386
chameleon => 9752
cherrypy => 34549
circuits => 23871
django => 233026
falcon => 6853
flask => 12479
jinja2 => 16665
pyramid => 55571
pysi => 1204
tornado => 28799
turbogears => 17037
web2py => 119402
webapp2 => 93549
webob => 17401
webpy => 12946
werkzeug => 30078
wheezy.caching => 3558
wheezy.core => 4043
wheezy.html => 3543
wheezy.http => 5643
wheezy.routing => 1819
wheezy.security => 1006
wheezy.template => 3381
wheezy.validation => 2920
wheezy.web => 7053

number of PEP8 errors or warnings:
bottle => 838
chameleon => 267
cherrypy => 1145
circuits => 103
django => 3168
falcon => 5
flask => 430
jinja2 => 438
pyramid => 3623
pysi => 186
tornado => 125
turbogears => 1080
web2py => 21224
webapp2 => 5174
webob => 1393
webpy => 880
werkzeug => 804
wheezy.caching => 0
wheezy.core => 0
wheezy.html => 0
wheezy.http => 0
wheezy.routing => 0
wheezy.security => 0
wheezy.template => 0
wheezy.validation => 0
wheezy.web => 0
Python has a number of web frameworks. A trivial PEP8 readability check gives you an idea where particular web framework stands in terms of internal quality. There is a wide field for improvement.

7 comments :

  1. "But most importantly: know when to be inconsistent -- sometimes the style guide just doesn't apply. When in doubt, use your best judgment. Look at other examples and decide what looks best."

    Not sure PEP8 is really a benchmark for quality, rather just a benchmark for conforming to a certain design.

    Although i do applaud your efforts to make wheezy conform, not really something that has to be boasted about given the above PEP8 paragraph.

    ReplyDelete
    Replies
    1. That should be an exception rather than rule, in my post I excluded 4 most controversial. If you believe there should be more `special` cases for exclusion, please let me know.

      Delete
    2. Would be interesting to see how Python itself conforms to PEP8...

      Delete
    3. Python 2.7.3 (maintenance only): 602678 lines, 81291 pep8 errors, 0.13 error ratio.
      Python 3.3.0 (future): 572,749 lines, 62,447 pep8 errors, 0.11 error ratio.

      Delete
    4. So Python's "internal quality" is bad?

      Delete
    5. I would say inconsistent, however the trend shows improvement in future versions.

      Delete
  2. I expect the main factor change the standard library's score is simply the fact that it is getting bigger, and new additions tend to be PEP 8 compliant, while older code often isn't (especially the parts the predate the existence of PEP 8).

    There's a very low return on updating existing code to be PEP 8 compliant just for the sake of it. We've only done it once that I can recall (for the threading module) and that was enough work to do in a backwards compatible way that we're unlikely to do it for any more.

    Since PEP 8 evolves over time, any existing code base with strong backwards compatibility rules needs to be judged against its own coding standards, not against the evolving standard for the standard library.

    ReplyDelete