CubicWeb has this really nice builtin facet system to define restrictions filters really as easily as possible.

We've just added two new kind of facets in CubicWeb :

• The RangeFacet which displays a slider using jquery to choose a lower bound and an upper bound. The RangeWidget works with either numerical values or date values
• The HasRelationFacet which displays a simple checkbox and lets you refine your selection in order to get only entities that actually use this relation.

Here's an example of code that defines a facet to filter musical works according to their composition date:

class CompositionDateFacet(DateRangeFacet):
    # 1. make sure this facet is displayed only on Track selection
    __select__ = DateRangeFacet.__select__ & implements('Track')
    # 2. give the facet an id (required by CubicWeb)
    id = 'compdate-facet'
    # 3. specify the attribute name that actually stores the date in the DB
    rtype = 'composition_date'

And that's it, on each page displaying tracks, you'll be able to filter them according to their composition date with a jquery slider.

All this, brought by CubicWeb (in the next 3.3 version)

In april a bunch of bugs have been corrected on the stable branch of cubicweb (3.1 series) and we've been working on the next generation series : 3.2. Here's a quick summary of what's been going on :

• cubicweb (the framework) was released twice with 3.1.3 and 3.1.4 which fixed a few bugs in the querier and the management screens
• cubicweb-blog 1.5.0 was released with some improvements to the graphical rendering
• cubicweb-tag 1.4.5 was released with notable improvements to tag clouds (added colors and better scaling of tags).
• cubicweb-file got a bugfix in 1.4.4
• cubicweb-mailinglist got a bugfix 1.3.1.

Next up, we are working on the 3.2.0 version of cubicweb with some particular focus on :

• form generation
• more explicit view registration (less magic)
• simpler workflow definitions
• js, css and ajax improvements

Do not hesitate to try the development branch (named tls-sprint at the moment) or read the changes at http://www.logilab.org/hg/cubicweb

The CubicWeb plateform will be on display at the French conference about linux "Solution Linux" hosted in Paris in the next 3 days. You can meet us at the System@tic stand or see us talk about it during a talk about Web2 this afternoon.

More info in french on the Logilab.org Blog.

If you feel that one of your pages takes more time than it should to be generated, chances are that you're making too many RQL queries. Obviously, there are other reasons but my personal experience tends to show this is first thing to track down. Luckily for us, CubicWeb provides a configuration option to log rql queries. In your all-in-one.conf file, set the query-log-file option:

# web application query log file
query-log-file=~/myapp-rql.log

Then restart your application, reload your page and stop your application. The file myapp-rql.log now contains the list of RQL queries that were executed during your test. It's a simple text file containing lines such as:

Any A WHERE X eid %(x)s, X lastname A {'x': 448} -- (0.002 sec, 0.010 CPU sec)
Any A WHERE X eid %(x)s, X firstname A {'x': 447} -- (0.002 sec, 0.000 CPU sec)

The structure of each line is:

<RQL QUERY> <QUERY ARGS IF ANY> -- <TIME SPENT>

Use the cubicweb-ctl exlog command to examine and summarize data found in such a file:

adim@crater:~$ cubicweb-ctl exlog < ~/myapp-rql.log
0.07 50 Any A WHERE X eid %(x)s, X firstname A {}
0.05 50 Any A WHERE X eid %(x)s, X lastname A {}
0.01 1 Any X,AA ORDERBY AA DESC WHERE E eid %(x)s, E employees X, X modification_date AA {}
0.01 1 Any X WHERE X eid %(x)s, X owned_by U, U eid %(u)s {, }
0.01 1 Any B,T,P ORDERBY lower(T) WHERE B is Bookmark,B title T, B path P, B bookmarked_by U, U eid %(x)s {}
0.01 1 Any A,B,C,D WHERE A eid %(x)s,A name B,A creation_date C,A modification_date D {}

This command sorts and uniquifies queries so that it's easy to see where is the hot spot that needs optimization.

Having said all this, it would probably be worth talking about the fetch_attrs attribute you can define in your entity classes because it can greatly reduce the number of queries executed but I'll make a specific blog entry for this.

I should finally mention the existence of the profile option in the all-in-on.conf. If set, this option will make your application run in an hotshot session and store the results in the specified file.

This article is part of the endless "you are never the only one experimenting with what sounds like a good idea". Just compare the following links:

The MIT Simile project produced the Exhibit mega-js-widget:

• Exhibit of the US presidents (click on the details tab)
• Exhibit of the MIT nobelists (click on the tabs)

Google ran an experiment with alternate views for search results:

• Location of PGA Tour tournaments
• Evolution of nanotechnologies over time
• Images in search results (click on Images on the right)

CubicWeb has built-in support for applying views to a selection of objects:

• Impressionism paintings in the museums of Normandy (click on the tabs)

You might have noticed here and there the mysterious user "mailbot" on cubicweb.org or logilab.org (both running the CubicWeb web app). Who is this user ?

Well, one of the cool features about cubicweb is that you can interact with it simply by using your email. When you are registered on a site, you can subscribe to a software project for example, from then on, you receive notifications of the new tickets and comments on the project. When you receive such a notification you can simply do an email reply to the new ticket or new comment, and cubicweb on the receiving end will import the content of your email to the website. When the content is imported that way, it's the mailbot doing the job.

This is not rocket science, but it sure is useful. Follow the activity of the site by email and interact directly with comments and tickets from your mail client!

image by husin.sani under creative commons

Today, I felt like doing a quick tour of the migration features provided by the ORMs used by the Python web frameworks. I started with Django. South looks better than Django-evolution which looks much better than dmigrations which is very low level. I also had a look at SQLAlchemy.migrate, but again, that's too low level for me since I am looking to define migrations with the same vocabulary that is used for the data model, independently of the underlying database schema.

The features listed in the South documentation have all been in CubicWeb for some time, except dependencies and autodetection. In my opinion, the dependency feature is not needed when you already have a list of scripts ordered by number, which is the case in South and in CubicWeb. The autodetection feature is more interesting, but it is tricky to get right. CubicWeb migration mechanism has had some kind of autodetection for a long time, but it is limited to the part that is easy to get right, yet quite common and useful:

• synchronizing properties of attributes and relationships (i.e. a Person.name becomes fulltextindexed or a has_portfolio relationship changes from 1-1 to 1-n)
• synchronizing permissions

For other common tasks like adding or removing entities and attributes, high-level directives are provided like add_entity_type or remove_attribute.

Up to now, not pushing autodetection of changes in the data model has been a deliberate choice, for diff'ing two models is complex and creating a migration path is even more difficult. Moreover, letting the ORM automatically overwrite local changes in the database schema can be harmful in some cases.

In CubicWeb, the idea is that the developer knows better than the framework, so let him decide what's best and provide him with a concise vocabulary to write the migration scripts.

photo by Tim in Sydney under creative commons.

There is this so-called 'gmap-view' in CubicWeb, the question is: how to use it ?

Well, first, no surprise, you have to generate an API key to be able to use google maps on your server (make sure your usage conforms the terms as defined by Google).

Now, let's say you have defined the following schema:

class Company(EntityType):
    name = String(required=True, maxsize=64)
    # ... some other attributes ...
    latitude = Float(required=True)
    longitude = Float(required=True)

class Employee(EntityType):
    # ... some attributes ...
    works_for = SubjectRelation('Company', cardinality='1*')

And you'd like to be able to display companies on a map; you've also got these nice icons that you'd wish to use as markers on the map. First thing, define those three icons as external resources. You can do that by editing your CUBE/data/external_resources file:

SMALL_MARKER_ICON=DATADIR/small_company.png
MEDIUM_MARKER_ICON=DATADIR/MEDIUM_company.png
BIG_MARKER_ICON=DATADIR/big_company.png

We're nearly done, now. We just have to make our entity class implement the cubicweb.interfaces.IGeocodable interface. Here's an example:

from cubicweb.entities import AnyEntity
from cubicweb.interfaces import IGeocodable

class Company(AnyEntity):
    id = 'Company' # this must match the type as defined in your schema
    __implements__ = AnyEntity.__implements__ + (IGeocodable,)

    def size(self):
        return self.req.execute('Any COUNT(E) WHERE E works_for C, C eid %(c)s',
                                {'c': self.eid})

    # this is a method of IGeocodable
    def marker_icon(self):
        size = self.size()
        if size < 20:
            return self.req_external_resource('SMALL_MARKER_ICON')
        elif size < 500:
            return self.req_external_resource('MEDIUM_MARKER_ICON')
        else:
            return self.req_external_resource('BIG_MARKER_ICON')

That's it, you can now call the gmap-view on a resultset containing companies:

rset = self.req.execute('Any C WHERE C is Company')
self.wview(rset, 'gmap-view', gmap_key=YOUR_API_KEY)

Further configuration is possible, especially to control the size of the map or the default zoom level.

To be fair, I must say that in a real-life cube, chances are you won't be able to specificy directly latitude and longitude and that you'll only have an address. This is slightly more complex to do since you'll need to query a geocoding service (the google one for instance) to transform your address into latitude/longitude. This will typically be done in a hook

Here is an screenshot of google maps on a production site, the museums in Normandy :

Here is a brief summary of what you get for the new CubicWeb 3.1.0 release. You could obviously go though the tickets on the version page, but here is the short version.

What new features ?

• a few OWL and Linked_Data functionalities
• navigation is now more complete on search results
• when installing a new cube that requires anonymous access (public site) the installer enables that access

What bugs are fixed ?

• a few things didn't work with opera and IE6
• json controller conflicts solved
• the newcube command is working again
• facets don't get in the way of the association process anymore
• and more...

Hope you enjoy this version... to see what's coming next, you can check out the planned versions of CubicWeb : 3.1.1 and 3.2.0.

Recently, for internal purposes, we've made a little cubicweb application to help us organizing visits to find new office locations. Here's an excerpt of the schema:

class Office(WorkflowableEntityType):
    price = Int(description='euros / m2 / HC / HT')
    surface = Int(description='m2')
    description = RichString(fulltextindexed=True)
    has_address = SubjectRelation('PostalAddress', cardinality='1?', composite='subject')
    proposed_by = SubjectRelation('Agency')
    comments = ObjectRelation('Comment', cardinality='1*', composite='object')
    screenshots = SubjectRelation(('File', 'Image'), cardinality='*1',
                                  composite='subject')

The two other entity types defined in the schema are Visit and Agency but we can also guess from the above that this application uses the two cubes comment and addressbook (remember, cubicweb is only a game where you assemble cubes !).

While we know that just defining the schema in enough to have a full, usable, (testable !) application, we also know that every application needs to be customized to fulfill the needs it was built for. So in this case, what we needed most was some custom filters that would let us restrict searches according to surfaces, prices or zipcodes. Fortunately for us, Cubicweb provides the facets (image) mechanism and a few base classes that make the task quite easy:

class PostalCodeFacet(RelationFacet):
    id = 'postalcode-facet'             # every registered class must have an id
    __select__ = implements('Office')   # this facet should only be selected when
                                        # visualizing offices
    rtype = 'has_address'               # this facet is a filter on the entity linked to
                                        # the office thrhough the relation has_address
    target_attr = 'postalcode'          # the filter's key is the attribute "postal_code"
                                        # of the target PostalAddress entity

This is a typical RelationFacet: we want to be able to filter offices according to the attribute postalcode of their associated PostalAdress. Each line in the class is explained by the comment on its right.

Now, here is the code to define a filter based on the surface attribute of the Office:

class SurfaceFacet(AttributeFacet):
    id = 'surface-facet'              # every registered class must have an id
    __select__ = implements('Office') # this facet should only be selected when
                                      # visualizing offices
    rtype = 'surface'                 # the filter's key is the attribute "surface"
    comparator = '>='                 # override the default value of operator since
                                      # we want to filter according to a minimal
                                      # value, not an exact one

    def rset_vocabulary(self, ___):
        """override the default vocabulary method since we want to hard-code
        our threshold values.
        Not overriding would generate a filter box with all existing surfaces
        defined in the database.
        """
        return [('> 200', '200'), ('> 250', '250'),
                ('> 275', '275'), ('> 300', '300')]

And that's it: we have two filter boxes automatically displayed on each page presenting more than one office. The price facet is basically the same as the surface one but with a different vocabulary and with rtype = 'price'.

(The cube also benefits from the builtin google map views defined by cubicweb but that's for another blog).