Blog entries

  • Migration in Python Web Frameworks ORMs

    2009/03/13 by Nicolas Chauvat

    Today, I felt like doing a quick tour of the migration features provided by the ORMs used by the Python web frameworks. I started with Django. South looks better than Django-evolution which looks much better than dmigrations which is very low level. I also had a look at SQLAlchemy.migrate, but again, that's too low level for me since I am looking to define migrations with the same vocabulary that is used for the data model, independently of the underlying database schema.

    http://south.aeracode.org/raw-attachment/wiki/Logo/logo-trac.png

    The features listed in the South documentation have all been in CubicWeb for some time, except dependencies and autodetection. In my opinion, the dependency feature is not needed when you already have a list of scripts ordered by number, which is the case in South and in CubicWeb. The autodetection feature is more interesting, but it is tricky to get right. CubicWeb migration mechanism has had some kind of autodetection for a long time, but it is limited to the part that is easy to get right, yet quite common and useful:

    • synchronizing properties of attributes and relationships (i.e. a Person.name becomes fulltextindexed or a has_portfolio relationship changes from 1-1 to 1-n)
    • synchronizing permissions
    http://farm2.static.flickr.com/1007/666142945_1d675bc2a7_m.jpg

    For other common tasks like adding or removing entities and attributes, high-level directives are provided like add_entity_type or remove_attribute.

    Up to now, not pushing autodetection of changes in the data model has been a deliberate choice, for diff'ing two models is complex and creating a migration path is even more difficult. Moreover, letting the ORM automatically overwrite local changes in the database schema can be harmful in some cases.

    In CubicWeb, the idea is that the developer knows better than the framework, so let him decide what's best and provide him with a concise vocabulary to write the migration scripts.

    photo by Tim in Sydney under creative commons.


  • Cubicweb 3.2 : what's new

    2009/06/03 by Aurelien Campeas
    http://farm4.static.flickr.com/3045/2585844966_05f617cd92_m.jpg

    Cubicweb has experienced a rather large shakeup. Some things needed major restructuration, and that is why you have been left with few releases in the past few weeks. All the cubes available at http://www.cubicweb.org/project have been updated accordingly.

    Version 3.2 brings us considerable improvements for:

    Form construction

    Cubicweb has had for long a nice system of forms smart enough to build themselves out of one cube's schema and some programmer-provided hints (or 'relation tags') to fine-tune things.

    It was not easy however to customize these forms nor to build new ones from scratch.

    So the new form systems draws from django-forms flexibility and style, keeping all the automatic goodness, and also make it quite easy now to build or customize forms at will.

    This is the area were backwards compatibility is mostly gone. Custom forms will have to be rewritten. Don't be angry about that, the forms overhaul was long overdue, and from now it will only move in small evolutionary, well-mannered steps.

    Relation tags

    Along with the form subsystem is the __rtags__ mechanism substantially updated and made more extensible. The __rtags__ were quite incorrectly attached to entities class at the ORM level instead of being related to views and forms. The cubicweb.web.uicfg module now provides a comprehensive catalog of relation tags instances allowing automatic forms and views customisation in a nicely declarative manner.

    Cubicweb 3.2 still remains compatible with the old __rtags__.

    View selection/filtering

    Cubiweb has also had for long a nice mechanism to filter views applicable to a given result set, the selector system. Various base classes were provided to hide selectors from the programmer and it had grown a little messy.

    Selectors now have a nicer declarative feeling and the framework does not try to hide them. Quite the opposite: writing, maintaining and using selectors is now a breeze, and the base classes are gone. More is less !

    However Cubicweb 3.2 remains backward compatible with the old selectors. Runtime warnings will help you track these and adapt as you see fit.

    Other features

    On the smaller features side, worth mentioning are:

    • new RichString attribute type in schema definitions, that simplifies format and encoding management,
    • inline relation edition is now possible (it was formerly limited to attributes) with 'reledit' view,
    • workflow definition has been simplified,
    • web/views has been somewhat cleanup up and reorganized,
    • automatic registration of app objects can now be switched to manual mode (no more hairy hard-to-debug registerer mechanism),
    • a generic SIOC view,
    • a view synthetizing permissions across a whole app.

    We hope you enjoy this release! The cubicweb development team.

    photo by jared under creative commons


  • Django, lessons learned in the world of startup companies

    2010/06/02 by Sandrine Ribeau

    I went to the BayPIGgies meeting last thursday. The talk of this session was led by the chief software architect of RubberCan, Barnaby Bienkowski. The idea was to explain why Django turns out to be the choice a lot of startups make when building their web applications.

    Governement 2.0

    http://assets.sunlightfoundation.com/site/3.0/images/sf_logo_trans.png

    The fact that Django is recommended by Sunlight Foundation is important. This foundation is a non-partisan, non-profit organization based in Washington, DC that focuses on the digitization of government data and the creation of tools and Web sites to make that data easily accessible for all citizens. This is part of what is called Governement 2.0. It is a neologism for attempts to apply the social networking and integration advantages of Web 2.0 to the practice of government (see E-Governement).

    It looks like the Sunlight Foundation recommends Django because it comes from the publishing industry. I am not sure what is so special about this, but I wish I could get more details on it, so please add your comments below.

    Since the CubicWeb's community is still small, we are not yet recommended by such a large foundation, but we'll make more effort to talk about it and try to expand our community.

    Geo-localization

    http://geodjango.org/images/globe.png

    These days, geo-localization is a big deal in most applications. On that matter, what Django has to offer is GeoDjango, that recently became part of the Django core. It is integrated with the ORM and has pre-generated SQL queries, but it is not optimized. It uses PostGIS, which adds support for geographic objects to the PostgreSQL object-relational database. GeoDjango strives to make it as simple as possible to create geographic web applications, like location-based services. Some of the features it provides are:

    • Extensions to Django’s ORM for the querying and manipulation of spatial data
    • Editing of geometry fields inside the administration panels
    • Loosely-coupled, high-level Python interfaces for GIS geometry operations and data formats.
    http://openstreetmap.org/images/osm_logo.png?1271689861

    OpenStreetMap is used for the backend. It provides geographic data for any part of the world. This is a nice feature and we should consider it for CubicWeb. What we provide so far is an interface IGeocodable with related views gmap-view, gmap-bubble, geocoding-json and gmap-legend. We do not query this data yet, we simply render them nicely in a Google Map. You can find the details on how to use it here.

    Online stores

    Numerous web applications are not only service or data providers, they sell something. Satchmo is the Django tool to easily build online stores. It provides a shopping cart framework with checkout using different payment modules such as Authorize.net, TrustCommerce, CyberSource, PayPal, Google Checkout or Protx.

    CubicWeb does not provide a component allowing to build an online store, it's not yet a domain we worked on. But I'd like to talk a bit about the cube cubicweb-shoppingcart. This cube defines shopping item and shopping cart, and enables to add items to the shopping cart. It defines type of shopping items and only those can be added to the shopping cart. Whereas Satchmo required to define categories and add items within a category, cubicweb-shoppingcart does not oblige to define categories. Creating shopping items is the only thing you need to do. That makes this component usable not only for online store. For example, we used this cube to manage Euroscipy registration fees reusing the generic schema of a "virtual" shopping cart and its related ressources (web widgets, validation hook, ...).

    Re-usable components

    http://pinaxproject.com/site_media/img/pinax_logo.png

    Pinax has a overall good satisfaction as it supports basics components for blogging, tagging, registration, notification and so on. But one point that was raised, is the difficulty of customizing Pinax components. It seems easy to write your own version of Pinax components, but to integrate them is a pain. All the components are tightly related and by customizing one, there is a big chance it will affect the other components.

    This last point is a big disadvantage. Why? Well, as a developer there is always something that you need to adjust to fit your needs. So customizing components is something you will not avoid while developing your web application. And something I'd like to point about CubicWeb, is its simplicity of re-using existing components, which are independent from each others. This is as easy as Python inheritance. And with its VRegistry, selectors and application objects (see The VRegistry, selectors and application objects for more details), customization is well integrated into the framework.

    Assemble cubes and functionalities is very easy as well. Let's think of an example. We have those three cubes: cubicweb-book, cubicweb-tag and cubicweb-comment. Cubicweb-book defines Book entity type. Cubicweb-tag defines Tag entities and the ability to tag other entity types. Cubicweb-comment defines Comment entity type and the ability to comment other entity types. What if we want to create an application in which we could tag and comment Book. Well, this is done with the following schema definition where we explicitly define the relations between Book, Tag and Comment entity types:

    from yams.buildobjs import RelationDefinition
    class comments(RelationDefinition):
        subject = 'Comment'
        object = 'Book'
        cardinality = '1*'
        composite = 'subject'
    
    class tag(RelationDefinition):
        subject = 'Tag'
        object = 'Book'
        cardinality = '**'
    

    Forms

    Despite the fact that forms are easy in Django, there is no way to add inline entities, at least for now (see this proposition) as easily as in CubicWeb (see HTML form construction for more details). That is very neat when you create/edit related entities. Plus, since CubicWeb 3.6, forms are much easier to handle, and we still put a lot of effort into making it simplier.

    So, yes, overall Django is selected as the best compromise, but for the reason I listed, CubicWeb should be considered.

    Watch out Django, we are getting on your way ;)