News about the framework and its uses.
The Logilab team holds a roadmap meeting every two months to plan its CubicWeb
development effort. Here is the report about the Sept 6th, 2013 meeting. The
previous report posted to the blog was the february 2013 roadmap.
This version is now stable and maintained (release 3.17.7 is upcoming). It added
a couple features and focused on putting CW to the diet by extracting some
functionnalities provided by the core into external cubes: sioc, embed,
massmailing, geocoding, etc.
For details read what's new in CubicWeb 3.17.
This version is now freezed and will be published as soon as all the patches are
tested and merged. Since we have a lot of work for clients until the end of the
year at Logilab, the community should feel free to help (as usual) if it wants
this version to be released rather sooner than later.
This version will remove the ldapuser source that is replaced by ldapfeed,
implement Cross Origin Resource Sharing, drop some very old compatibility code,
deprecate the old version of the multi-source system and provide various other
features and bugfixes.
For details read list of tickets for CubicWeb 3.18.0.
This version will probably be publish early next year (read january or
february 2014) unless someone who is not working at Logilab takes responsibility
for its release.
It should include the heavy refactoring work done by Pierre-Yves and Sylvain
over the past year, that modifies sessions and sources to lay the path for
For details read list of tickets for CubicWeb 3.19.0 or take a look at this head.
Since Orbui changes the organization of the default user interface on screen, it
was decided to share the low-level bootstrap related views that could be shared
and build a SquareUI cube that would conform design choices of the default UI.
Logilab is now developping all its new projects based on Squareui 0.2. Read
about it on the mailing list archives.
The mid-term goals include better REST support (Representational State
Transfer), complete WSGI (Python's Web Server Gateway Interface) and the FROM
clause for RQL queries (to reinvent db federation outside of the core).
Our current plan is to extract as much as possible to cubes. We started CubicWeb
many years ago with the Python motto "batteries included", but have since
realized that having too much in the core contributes to making CubicWeb
difficult to learn.
Since we would very much like the community to grow, we are now aiming for
something more balanced, like Mercurial does. The core is designed such that
most features can be developed as an extension. Once they are stable, popular
extensions can be moved to the main library that is distributed with the core,
and be activated with a switch in the configuration file.
Several cubes are under active development: oauth, signedrequest, dataio, etc.
As already said on the mailing list, other developers and contributors are more
than welcome to share their own goals in order to define a roadmap that best
fits everyone's needs.
Logilab's next roadmap meeting will be held at the beginning of November 2013.
What's new in CubicWeb 3.17?
- add a command to compare db schema and file system schema
- Add CubicWebRequestBase.content with the content of the HTTP request
- Add directive bookmark to ReST rendering
- Allow user defined final type
- drop typed_eid() in favour of int() (see #2742462)
- The SIOC views and adapters have been removed from CubicWeb and moved to the
- The web page embedding views and adapters have been removed from CubicWeb and
moved to the embed cube.
- The email sending views and controllers have been removed from CubicWeb and
moved to the massmailing cube.
- RenderAndSendNotificationView is deprecated in favor of
ActualNotificationOp the new operation uses the more efficient data
- Looping task can now have an interval <= 0. Negative interval disable the
looping task entirely.
- We now serve html instead of xhtml.
- ldapuser has been deprecated. It will be removed in a future
version. If you are still using ldapuser switch to ldapfeed NOW!
- hijack_user has been deprecated. It will be dropped soon.
- The progress views and adapters have been removed from CubicWeb. These
classes were deprecated since 3.14.0. They are still available in the
- The part of the API deprecated since 3.7 was dropped.
A few people of the CubicWeb team are going to attend the French PostgreSQL community conference in Nantes (France) on the 13th of june.
We're excited to learn more about the following topics that are relevant to CubicWeb's development and features :
Obviously we'll pay attention to all the talks during the day. If you're attending, we hope to see you there.
With a few people from Logilab we went to the 2nd International Workshop on Open Data (WOD), on the 3rd of june.
Although the main focus was an academic take on OpenData, a lot of talks were related to the Semantic Web technologies and especially LinkedData.
The full program (and papers) is on the following website. Here is a quick review of the things we though worth sharing.
- privacy oriented ontologies : http://l2tap.org/
- interesting automations done to suggest alignments when initial data is uploaded to an opendata website
- some opendata platforms have built-in APIs to get files, one example is Socrata : http://dev.socrata.com/
- some work is being done to scale processing of linked data in the cloud (did you know you could access ready available datasets in the Amazon cloud ? DBPedia for example )
- the data stored in wikipedia can be a good source of vocabulary on certain machine learning tasks (and in the future, wikidata project)
- there is an RDF extension to Google Refine (or OpenRefine), but we haven't managed to get it working out of the box,
- WebSmatch uses morphological operators (erosion / dilation) to identify grids and zones in Excel Spreadsheets and then aligns column data on known reference values (e.g. country lists).
We naturally enjoyed the presentation made by Romain Wenz about http://data.bnf.fr with the unavoidable mention of Victor Hugo (and CubicWeb).
Thanks to the organizers of the conference and to the National French Library for hosting the event.
CubicWeb got rewarded yesterday at the award ceremony of the Dataconnexions 2013 contest.
Dataconnexions is a contest organized by Etalab, the organization part of the French State that is in charge of data.gouv.fr, that catalogs the open data published by the french administration.
Congratulations to all the developers and users of CubicWeb and welcome to the people who will join the CW community thanks to the media coverage we are now experiencing.
Read the announce to the press and the slides.
The Logilab team now holds a roadmap meeting every two months to plan its CubicWeb development effort. Here are the decisions that were taken on Feb 1st, 2013.
This version should be published before the end of March and will finish all the things
that are work in progress. It will include:
- the refactoring necessary to introduce persistant sessions,
- the shrinking of web/views: everything that does not deserve its own cube (like sioc, embed,
geocoding, etc) will go into a cube named legacyui (this will open the door to squareui),
- stop serving pages with "content-type: application/xhtml",
- handling postgresql schemas (will require a new version of logilab.database),
- a new logo.
Once the cube legacyui extracted (in version 3.17), it will be possible to move forward
swiftly with squareui. Due to its other duties, one can not expect the core CW team to develop squareui.
People interested will be in charge and ideally the squareui cube could be released when cubicweb 3.17
will be published.
The lead CW developers will spend about 20% of their time cleaning up the ticket backlog at the forge (900 open tickets and 50 in progress !)
The first step will be to reduce the number of tickets "in progress", then to organize the open tickets and merge the duplicates.
This version is due at the end of may 2013. It will include:
- persisting sessions,
- RESTfulness: support for HTTP verbs PUT / DELETE, enforcement of the semantics of
GET / POST (may be difficult to maintain backward-compatibility)
The mid-term goals are:
possibility to add new base types (Array, HStore,
Geometry, TSVector, etc.) that would use extensions from the
FROM clause in rql queries
defining attribute on relations and defining "virtual" relations or rules:
author = SubjectRelation('Person', cardinality='1*', inlined=True)
book = SubjectRelation('Book', cardinality='1*', inlined=True)
role = SubjectRelation('Role', cardinality='1*', inlined=True)
preface_writer = VirtualRelation('C is Contribution, C author S, C book O, '
'C role R, R name "preface writer"')
Any P WHERE B is Book, P preface_writer B
Will we need a materialized view in the database, a standard relation maintained by hooks,
rewrite the RQL on-the-fly ? Time will tell.
cards with logic (mustache js templates for example)
package separately Cubic(Web)DB et CubicWeb ?
think about the overall architecture (using WSGI, persistent sessions, etc.), and find
solutions that fit a distributed architecture (look at paste.deploy, circus, etc.)
configurable metadata, managing the size of the entities table
namespaces for the data models of the cubes
As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.
Logilab's next roadmap meeting will be held at the beginning of April 2013.
What's new in CubicWeb 3.16?
- Add a new dataimport store (SQLGenObjectStore). This store enables a fast
import of data (entity creation, link creation) in CubicWeb, by directly
flushing information in SQL. This may only be used with PostgreSQL, as it
requires the 'COPY FROM' command.
Orm: set_attributes and set_relations are unified (and
deprecated) in favor of cw_set that works in all cases.
db-api/configuration: all the external repository connection information is
now in an URL (see #2521848),
allowing to drop specific options of pyro nameserver host, group, etc and fix
broken ZMQ source. Configuration related changes:
- Dropped 'pyro-ns-host', 'pyro-instance-id', 'pyro-ns-group' from the client side
configuration, in favor of 'repository-uri'. NO MIGRATION IS DONE,
supposing there is no web-only configuration in the wild.
- Stop discovering the connection method through repo_method class attribute
of the configuration, varying according to the configuration class. This is
a first step on the way to a simpler configuration handling.
DB-API related changes:
- Stop indicating the connection method using ConnectionProperties.
- Drop _cnxtype attribute from Connection and cnxtype from
Session. The former is replaced by a is_repo_in_memory property
and the later is totaly useless.
- Turn repo_connect into _repo_connect to mark it as a private function.
- Deprecate in_memory_cnx which becomes useless, use _repo_connect instead
the "tcp://" uri scheme used for ZMQ
communications (in a way reminiscent of Pyro) is now named
"zmqpickle-tcp://", so as to make room for future zmq-based lightweight
communications (without python objects pickling).
Request.base_url gets a secure=True optional parameter that yields
an https url if possible, allowing hook-generated content to send
secure urls (e.g. when sending mail notifications)
Dataimport ucsvreader gets a new boolean ignore_errors
- Drop of cubicweb.web.uicfg.AutoformSectionRelationTags.bw_tag_map,
deprecated since 3.6.
- The RQL search bar has now some auto-completion support. It means
relation types or entity types can be suggested while typing. It is
an awesome improvement over the current behaviour !
- The action box associated with table views (from tableview.py)
has been transformed into a nice-looking series of small tabs; it
means that the possible actions are immediately visible and need not
be discovered by clicking on an almost invisible icon on the upper
- The uicfg module has moved to web/views/ and ui configuration
objects are now selectable. This will reduce the amount of
subclassing and whole methods replacement usually needed to
customize the ui behaviour in many cases.
- Remove changelog view, as neither cubicweb nor known cubes/applications
were properly feeding related files.
- 'pyrorql' sources will be automatically updated to use an URL to locate the source
rather than configuration option. 'zmqrql' sources were broken before this change,
so no upgrade is needed...
- Debugging filters for Hooks and Operations have been added.
- Some cubicweb-ctl commands used to show the output of msgcat and
msgfmt; they don't anymore.