Blog entries

  • CubicWeb 3.7 released

    2010/03/19

    Hi there !

    I'm pleased to announce the 3.7 release of CubicWeb, after a much shorter development cycle than for the 3.6...

    But it still have some interesting changes:

    • NOW DEPENDS ON PYTHON 2.5
    • use the newly created logilab.database package (you'll have to install it as well as upgrade logilab.common and rql)
    • proper behaviour on the repository side of cubiweb:
      • dropped unsafe_execute, execute is now unsafe by default in hooks and operations. You can still explicitly control security using the enabled_secury context manager
      • proper transaction hooks control using the hooks_control context manager
    • started some transaction undo support (only undo of deletion supported right now)
    • various other bug fixes and improvments

    Notice the 3.6 branch will still be maintained for some time.

    Enjoy!


  • MS SQL Server backuping gotcha

    2010/01/19

    While working on the port of CubicWeb to the Windows platform, including supporting MS Sql Server as the database backend, I got bitten by a weird behavior of that database engine. When working with cubicweb, most administrations command are wrappped by the cubicweb-ctl utility and database backups are performed by running cubicweb-ctl db-dump <instancename>. If the instance uses PostgreSQL as the backend, this will call the pg_dump utility.

    When porting to Sql Server, I could not find such a utility, but I found that Transact SQL has a BACKUP DATABASE command, so I was able to call it using Python's pyodbc module. I tested it interactively, and was satisfied with the result:

    >>> from logilab.common.db import get_connection
    >>> cnx = get_connection(driver='sqlserver2005', database='mydb', host='localhost', extra_args='autocommit;trusted_connection')
    >>> cursor = cnx.cursor()
    >>> cursor.execute('BACKUP DATABASE ? TO DISK = ?', ('mydb', 'C:\\Data\\mydb.dump'))
    >>> cnx.close()
    

    However, testing that very same code through cubicweb-ctl produced no file in C:\\Data\\. To make a (quite) long story short, the thing is that the BACKUP DATABASE command is asynchronous (or maybe the odbc driver is) and the call to cursor.execute(...) will return immediately, before the backup actually starts. When running interactively, by the time I got to type cnx.close() the backup was finished but when running in a function, the connection was closed before the backup started (which effectively killed the backup operation).

    I worked around this by monitoring the size of the backup file in a loop and waiting until that size gets stable before closing the connection:

    import os
    import time
    from logilab.common.db import get_connection
    
    filename = 'c:\\data\\toto.dump'
    dbname = 'mydb'
    cnx = get_connection(driver='sqlserver2005',
                         host='localhost',
                         database=dbname,
                         extra_args='autocommit;trusted_connection')
    cursor = cnx.cursor()
    cursor.execute("BACKUP DATABASE ? TO DISK= ? ", (dbname, filename,))
    prev_size = -1
    err_count = 0
    same_size_count = 0
    while err_count < 10 and same_size_count < 10:
        time.sleep(1)
        try:
            size = os.path.getsize(filename)
            print 'file size', size
        except OSError, exc:
            err_count +=1
            print exc
        if size > prev_size:
            same_size_count = 0
            prev_size = size
        else:
           same_size_count += 1
    cnx.close()
    

    I hope sharing this will save some people time...

    Note: get_connection() comes from logilab.common.db which is a wrapper module which tries to simplify writing code for different database backends by handling once for all various idiosyncrasies. If you want pure pyodbc code, you can replace it with:

    from pyodbc import connect
    cnx = connect(driver='SQL Server Native Client 10.0',
                  host='locahost',
                  database=dbname,
                  trusted_connection='yes',
                  autocommit=True)
    

    The autocommit=True part is especially important, because BACKUP DATABASE will fail if run from within a transaction.


  • CubicWeb 3.9 released

    2010/07/12 by Sylvain Thenault

    CubicWeb 3.9.0 went out last week. We now have tested it in production and fixed the remaining bugs, which means it is now show time!

    http://www.cubicweb.org/file/1179905?vid=download

    What's new in CubicWeb 3.9?

    The 3.9 release development was started by a one week long sprint at the beginning of May. The two goals were first to make it easier to customize the look and feel of a CubicWeb application, and second to do a big cleanup of the javascript library. This led to the following major changes.

    • We introduced property sheets, which replace former external_resources file, as well as define some constants that will be used to 'compile' cubicweb and cubes' stylesheets.
    • We started a new, clean cubicweb.css stylesheet, that tries to keep up with the rhythm. This is still a work in progress, and by default the old css is still used, unless specified otherwise in the configuration file.
    • We set the bases for web functional testing using windmill. See test cases in cubicweb/web/test/windmill/ and python wrapper in cubicweb/web/test_windmill/ if you want to use this in your own cube.
    • We set the bases for javascript unit-testing using qunit. See test cases in cubicweb/web/test/jstests/ and python wrapper in cubicweb/web/test_jscript/ if you want to use this in your own cube.
    • We cleaned the javascript code: the generic stuff moved into the cw namespace, the ajax api is now much simpler thanks to more generic and powerful functions. As usual backward compatibility was kept, which means that your existing code will still run, but you will see tons of deprecation warnings in the firebug console.
    • We implemented a simple documentation extraction system for javascript. Just put ReST in javascript comments, and get all the power of sphinx for documenting your javascript code.

    But that's not all! There are also two major changes in 3.9.

    http://www.cubicweb.org/file/1179904?vid=download

    Architectural change: adapters

    The first major change is the introduction of adapters, also found in the Zope Component Architecture and documented in the GoF book. This will allow for better application design and easier code reuse. You can see several usage in the framework, for instance the "ITree" adapter in cubicweb.entities.adapters, the "IBreadCrumbs" adapter in cubicweb.web.views.ibreadcrumbs, or still the "ICalendarable" adapter in cubicweb.web.views.calendar.

    Important full search improvement

    The second major change will benefit directly to end users: we worked with our friends from SecondWeb to expose the ranking feature found in postgres full-text search. This clearly improves the user experience when doing full-text searches. Ranking may be finely tuned by setting different weights to entity types, entity types attributes, or even be dynamically computed per entity instance. Of course, all this is done in an adapter, see "IFTIndexableAdapter" in cubicweb/entities/adapters.py.

    Minor changes

    Other minor changes include:

    • support for wildcard text search for application using postgres >= 8.4 as backend. Try searching for 'cub*' on cubicweb.org for instance.
    • inline edition of composite relation
    • nicer, clickable, schema image of the data model
    • enhanced support for the SQLserver database

    Enjoy!


  • CubicWeb 3.8 released

    2010/04/28 by Sylvain Thenault

    CubicWeb 3.8.0 went out last week, but now we have tested it, produced a 3.8.1, it's show time!

    What's new in CubicWeb 3.8?

    One of the most important change is http server update to move from deadend twisted.web2 to twisted.web. With this change comes the possibility to configure the maximum size of POST request in the configuration file (was hard-coded to 100Mo before).

    Other changes include:

    • CubicWeb should now be installable through pip or easy_install. This is still experimental, and we don't use it that much so please, give us some feedback! Some cubes are now also "pipable" (comment, blog...), but more will come with new releases.
    • .execute() function lost its cache key argument. This is great news since it was a pain to explain and most cubicweb users didn't know how to handle it well (and I'm thre greatest beneficer since I won't have to explain over and over again)
    • nicer schema and workflow views
    • refactored web session handling, which should now be cleaner, clearer, hence less buggy...
    • nicer skeleton generation for new cubes, cleaner __pkginfo__ (you don't have to define both __depends__ / __depends_cubes__ or __recommends__ / __recommends_cubes__ in the general case, and other cleanups)

    Enjoy!