Category Archives: Development

Why Tornado Framework?

In our last post (http://mercenarytech.com/2013/07/why-django-framework/) we covered Django framework. Django is a fine choice, with module and configuration organization as its biggest strength. If it fits your requirements then it is a safe choice; it’s certainly popular. But, my experience has been that in practice it doesn’t actually offer much benefit. When given a choice (and based on project requirements) I push customers towards Tornado Framework. In this post I will talk about the reasons.

A Little Strategy
When I evaluate application frameworks technically, I’m generally looking for:

1) Features that will buy time or at least not cost. Sometimes simplicity is better than sophistication.

2) Assumption on the part of the framework architect that do not fit my roadmap. These become the corners we paint ourselves into.

3) Deployment or user experience compromises. Are there any the framework is asking me to make? Again, see point #2.

Thus, when I’m thinking about the future technical direction of an application, features like automatically generating template code, say for a Controller class, are interesting, but that is something I can automate from an IDE. It may not buy me much as a part of an application framework. Sure, if I can save 30 seconds by generating boilerplate code, especially if I will potentially write that same boilerplate code 1000 times in the life of that project is useful, but in part I judge a framework on not having to write that boilerplate at all. 

Simplicity Matters

Simplicity counts for a lot in my book. Take Controllers again. A Java framework like Spring contains many types of Controllers, where you have to learn something about each kind in order to subclass the right one. Tornado contains two – one for http, one for websockets. The User model in Django is another example of this. Depending on your version of Django, the process for extending the User model is different and requires understanding the internals of Django to correctly modify. With Tornado, there is a get_current_user(), you simply override it with whatever representation of user you wish – could be as simple as a cookie.

Where Django and Flask don’t make sense to me is they seem to assume I’m building a web application that will serve up html pages, and in the case of Django, have a significant administrative component. There’s a further assumption that my application will be monolithic in nature and that I need an MVC framework tightly coupled to a persistence layer. This is not how applications are being constructed today. Applications have moved on from that model, with real-time push technology (like websockets) becoming commonplace, and ever more functionality pushed to the client side. Thus web applications that a generation ago would have rendered pages from the server are instead feeding json from the server to a user experience rendered entirely in the browser, which in turn is more commonly a mobile device. Django and Flask are not offering anything unique to this style of application apart from a convenient way to organize your source code.

The Io loop / event model in Tornado is sadly overlooked. Tornado’s detractors would claim the moment you have to do something concurrent (fetching from a database is often used as an example), you lose the benefit. This is a false assumption though, for three reasons:

1) There are asynchronous drivers available for many databases and queues.
2) This assumes a monolithic stack, where the app needs to make a direct connection to the database (likely false as you scale)
3) This ignores real data (google it) that Tornado is significantly more performant than Apache with wsgi (for example).

On the last point, I’d challenge that setting up apache with wsgi is not difficult, but is more effort than Nginx with Tornado, for lesser performance. So why choose Django or Flask when that’s the standard deployment practice?

Tornado provides a basic template system, internationalization, simple organization, and python itself gives you parsing (json, XML,etc) and logging, no framework needed. SQLAlchemy is at least as full-featured as Django ORM and integrates well. On top of this you get a high performance http server and mature websockets support. Thrown in a built-in asynchronous http client and you have a framework that doesn’t get in your way but does give you the pieces to build functionality quickly. Thus for me, the power of Tornado over other frameworks is that it offers just enough and does not make any assumptions about the kind of applications I can build.

The Problem with APM / NPM / TPM / OA (and Everything Else Your Vendor Might Call It)

Here’s the question that derails Application Performance Management projects:

“Who is going to use it?”

It’s the people, man! Dynatrace, Introscope, OpNet, Diagnostics, ExtraHop – sure there are differences among products, but the single common difference between success and failure is the person using the tool.

It’s really simple:

    1. If you buy tools, but don’t use them… they aren’t solving any of your problems.
    2. If you buy sophisticated technical tools but don’t have the prerequisite background or invest in training… they might as well be doorstops.
    3. If other teams use the tool but don’t share the information learned… nobody benefits (silos are bad)

Unfortunately, I’ve seen too many customers over the years make major investments in these tools – licensing, professional services, etc – and then ultimately look around and realize that after the consultants leave, there’s no one who can actually use them.

Some might say that Predictive Analytics and Automation will solve that – just put the brains in the software – but we aren’t there yet. With each new vendor into the market (New Relic, AppDynamics, for example), there is a focus on building a better mousetrap – more technology, more metrics. This is not making it any easier on Corporate IT, which is already cash-strapped and down to minimal staffing. For now, the best tool really is the one that will get used.

 

Why Django Framework?

Django is arguably the most popular python framework at the time of this writing. This popularity is oddly misplaced and in this post we’ll explore where Django is not offering the presumed panacea.

Django began life as a framework for online publishing of a news paper. In that domain, there was a real need for editors and publishers to have admin features that gave them control over the content of the published site. From my own experience I know that media application are often “widget sized” – a sidebar to highlight a story and add context, maybe a small game, a survey, often made under a tight deadline to coincide with a breaking news story. In short, limited customer interaction, limited transactional processing, and a focus on publishing HTML pages. If this is your domain then Django could be a good choice.

Administration
The automatic creation of an administration back end is one of the features that first popularized Django. In practice though, if your requirements go beyond basic create-read-update-delete of models you will need to develop your own administrative functionality – Django becomes moot.

Form Processing
Django includes capabilities for form processing. It’s fairly rigid, where the data types for fields are defined, along with error messages. Classic request/response page stuff. In an era where greater dynamic feedback is expected in the user interface, Django isn’t adding any real benefit.

Account Management
While Django provides a default User model type, in practice it must be extended for all but the simplest applications. Oddly, there is no default functionality for registering users or user login. There is an open-source registration application that can be integrated, but unfortunately it has not been consistently maintained. My experience has shown that the limited account management functionality is a throw-away.

Configuration
Django provides a single settings file for configuring a project. This is great for keeping project information organized. Unfortunately, it overlooks that different configuration is likely needed for different environments – development, test, production, etc.

Development Tools
Django provides scripts and a basic web server that make for a very responsive development workflow. Being able to regenerate a database schema from a single command is ideal. While there are commands to generate MVC components, in reality very little is actually generated – all the wiring and functionality is still in your hands to create.

Django ORM
Django includes its own object-relational mapping. There’s nothing obviously wrong with it, but as SQL Alchemy is more popular in the Python community, it isn’t really offering Django any unique advantage.

Python Templates
Python frameworks have taken a nearly identical approach in exposing Python code via HTML, with simplified tags for looping and conditional logic. Much like Tornado Framework, there is support for internationalization and localization via externalized text strings.

Summary
There’s nothing really wrong with Django. It organizes your code and configuration intelligently. Unfortunately, in practice it also doesn’t actually offer much benefit. Code generation and out-of-box functionality is really limited, especially for developers who may have seen other frameworks (Ruby on Rails and even Cocoa / xcode come to mind).

Why I Prefer Tornado Framework
We could have written this same post about Flask or any number of other Python frameworks. There’s nothing inherently wrong with most and any assumptions in the framework you can work around. So why do I prefer to use Tornado Framework, especially given its reputation for being bare-bones? That will be a future post.

Why I rejected Angularjs (and started using knockout)

We considered Angularjs for a recent project but rejected it for three reasons:
1) The routing mechanism produces ugly urls (not REST / human / search engine friendly)
2) The routing mechanism is required, but there is no capacity for authentication or roles associated with the routing.
3) If you want to use other JavaScript frameworks, such as d3 or rickshaw for other types of data visualization, you are forced to write an integration / wrapper layer to keep Angularjs happy.

There appears to be a strong sense among the Angularjs community that “one page applications are the future of web development.” I could not disagree with this sentiment more strongly and it is one of the things that drove me away, despite a general sense that it is a well constructed framework.

Ultimately what I was looking for on this particular project was a simple way to organize the front end of the application. Knockoutjs has proven to be exactly that – an unobtrusive framework for pulling data from the server and binding it to display elements. I can continue to use the more robust routing capabilities of my chosen server-side application framework (Tornado) and don’t have to spend time writing integration / wrapper code to use our chosen visualization frameworks either.

Nginx and Angularjs routes

Angularjs is quickly gaining traction as a javascript framework. One noticeable problem is the ugly (read: not SEO or human friendly) urls exposed through its routing mechanism. For example, a typical url ends up looking like /index.html#/foo. A “prettier” way to expose this would be as simple /foo. Nginx is a very popular web server and while there are a million examples of simple http 301 redirects and handling php files, there weren’t great examples for this simple scenario. So, here it is in all its glory, tested on nginx 1.2.6:

Add a block like the following to nginx.conf:

location =/foo {
rewrite ^ /index.html#/foo permanent;
}

One downside is that you’ll likely need a block like this for each url you want mapped. In theory one could write a regular expression that would take the uri and append it to the end of the rewrite, but in practice I couldn’t get it to work. If the information on the web is to be believed, it is slightly better performance to do an explicit match (in this case to exactly “/foo”) than to do the regex comparison.