subscribe to this blog

CubicWeb Blog

News about the framework and its uses.

Logilab's roadmap for CubicWeb on September 6th, 2013

2013/09/17 by Nicolas Chauvat

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. Here is the report about the Sept 6th, 2013 meeting. The previous report posted to the blog was the february 2013 roadmap.

Version 3.17

This version is now stable and maintained (release 3.17.7 is upcoming). It added a couple features and focused on putting CW to the diet by extracting some functionnalities provided by the core into external cubes: sioc, embed, massmailing, geocoding, etc.

For details read what's new in CubicWeb 3.17.

Version 3.18

This version is now freezed and will be published as soon as all the patches are tested and merged. Since we have a lot of work for clients until the end of the year at Logilab, the community should feel free to help (as usual) if it wants this version to be released rather sooner than later.

This version will remove the ldapuser source that is replaced by ldapfeed, implement Cross Origin Resource Sharing, drop some very old compatibility code, deprecate the old version of the multi-source system and provide various other features and bugfixes.

For details read list of tickets for CubicWeb 3.18.0.

Version 3.19

This version will probably be publish early next year (read january or february 2014) unless someone who is not working at Logilab takes responsibility for its release.

It should include the heavy refactoring work done by Pierre-Yves and Sylvain over the past year, that modifies sessions and sources to lay the path for CubicWeb 4.

For details read list of tickets for CubicWeb 3.19.0 or take a look at this head.

Squareui

Since Orbui changes the organization of the default user interface on screen, it was decided to share the low-level bootstrap related views that could be shared and build a SquareUI cube that would conform design choices of the default UI.

Logilab is now developping all its new projects based on Squareui 0.2. Read about it on the mailing list archives.

Mid-term goals

The mid-term goals include better REST support (Representational State Transfer), complete WSGI (Python's Web Server Gateway Interface) and the FROM clause for RQL queries (to reinvent db federation outside of the core).

Cubes

Our current plan is to extract as much as possible to cubes. We started CubicWeb many years ago with the Python motto "batteries included", but have since realized that having too much in the core contributes to making CubicWeb difficult to learn.

Since we would very much like the community to grow, we are now aiming for something more balanced, like Mercurial does. The core is designed such that most features can be developed as an extension. Once they are stable, popular extensions can be moved to the main library that is distributed with the core, and be activated with a switch in the configuration file.

Several cubes are under active development: oauth, signedrequest, dataio, etc.

Last but not least

As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.

Logilab's next roadmap meeting will be held at the beginning of November 2013.


Brainomics - A management system for exploring and merging heterogeneous brain mapping data

2013/09/12 by Arthur Lutz

At OBHM 2013, the 19th Annual Meeting of the Organization for Human Brain Mapping, Logilab presented a poster which explains the work done using CubicWeb on brain imaging and genetics data in collaboration with INRIA, INSERM and the CEA during the Brainomics project co-financed by Agence nationale de la Rercherche.

http://www.cubicweb.org/file/3123353/raw/Screenshot%20from%202013-09-12%2010%3A27%3A27.png

You can download this poster and try the demo online.


What's new in CubicWeb 3.17

2013/06/21 by Aurelien Campeas

What's new in CubicWeb 3.17?

New functionalities

  • add a command to compare db schema and file system schema (see #464991)
  • Add CubicWebRequestBase.content with the content of the HTTP request (see #2742453)
  • Add directive bookmark to ReST rendering (see #2545595)
  • Allow user defined final type (see #124342)

API changes

  • drop typed_eid() in favour of int() (see #2742462)
  • The SIOC views and adapters have been removed from CubicWeb and moved to the sioc cube.
  • The web page embedding views and adapters have been removed from CubicWeb and moved to the embed cube.
  • The email sending views and controllers have been removed from CubicWeb and moved to the massmailing cube.
  • RenderAndSendNotificationView is deprecated in favor of ActualNotificationOp the new operation uses the more efficient data idiom.
  • Looping task can now have an interval <= 0. Negative interval disable the looping task entirely.
  • We now serve html instead of xhtml. (see #2065651)

Deprecation

  • ldapuser has been deprecated. It will be removed in a future version. If you are still using ldapuser switch to ldapfeed NOW!
  • hijack_user has been deprecated. It will be dropped soon.

Deprecated Code Drops

  • The progress views and adapters have been removed from CubicWeb. These classes were deprecated since 3.14.0. They are still available in the iprogress cube.
  • The part of the API deprecated since 3.7 was dropped.

We're going to PGDay France, the Postgresql Community conference

2013/06/11 by Arthur Lutz

A few people of the CubicWeb team are going to attend the French PostgreSQL community conference in Nantes (France) on the 13th of june.

http://www.cubicweb.org/file/2932005/raw/hdr_left.png

We're excited to learn more about the following topics that are relevant to CubicWeb's development and features :

https://www.pgday.fr/_media/pgfr2.png

Obviously we'll pay attention to all the talks during the day. If you're attending, we hope to see you there.


OpenData meets the Semantic Web at WOD2013

2013/06/10 by Arthur Lutz

With a few people from Logilab we went to the 2nd International Workshop on Open Data (WOD), on the 3rd of june.

Although the main focus was an academic take on OpenData, a lot of talks were related to the Semantic Web technologies and especially LinkedData.

http://www.logilab.org/file/144837/raw/banniere-wod2013.png

The full program (and papers) is on the following website. Here is a quick review of the things we though worth sharing.

  • privacy oriented ontologies : http://l2tap.org/
  • interesting automations done to suggest alignments when initial data is uploaded to an opendata website
  • some opendata platforms have built-in APIs to get files, one example is Socrata : http://dev.socrata.com/
  • some work is being done to scale processing of linked data in the cloud (did you know you could access ready available datasets in the Amazon cloud ? DBPedia for example )
  • the data stored in wikipedia can be a good source of vocabulary on certain machine learning tasks (and in the future, wikidata project)
  • there is an RDF extension to Google Refine (or OpenRefine), but we haven't managed to get it working out of the box,
  • WebSmatch uses morphological operators (erosion / dilation) to identify grids and zones in Excel Spreadsheets and then aligns column data on known reference values (e.g. country lists).

We naturally enjoyed the presentation made by Romain Wenz about http://data.bnf.fr with the unavoidable mention of Victor Hugo (and CubicWeb).

Thanks to the organizers of the conference and to the National French Library for hosting the event.


data.bnf.fr gets the Stanford Prize for Innovation in Research Libraries

2013/03/01 by Nicolas Chauvat

data.bnf.fr and Gallica just got awarded the Stanford Prize for Innovation in Research Libraries 2013. The CubicWeb community is very pleased to see that data.bnf.fr, which is built with CubicWeb, is being recognized at the top international level as leading innovation its domain! Read the comments of the judges for more details.


CubicWeb at Data Tuesday on Feb 26th 2013

2013/02/15 by Nicolas Chauvat

CubicWeb was showcased at Data Tuesday on Feb 26th 2013. The other presentations were interesting, especially shacache.org, the soon-to-be-launched OpenMeteoData and the very useful scikit.learn.


CubicWeb rewarded at Dataconnexion 2013

2013/02/06 by Nicolas Chauvat

CubicWeb got rewarded yesterday at the award ceremony of the Dataconnexions 2013 contest.

http://www.cubicweb.org/2710848?vid=download

Dataconnexions is a contest organized by Etalab, the organization part of the French State that is in charge of data.gouv.fr, that catalogs the open data published by the french administration.

Congratulations to all the developers and users of CubicWeb and welcome to the people who will join the CW community thanks to the media coverage we are now experiencing.

Read the announce to the press and the slides.


Logilab's roadmap for CubicWeb as of February 2013

2013/02/04 by Nicolas Chauvat

The Logilab team now holds a roadmap meeting every two months to plan its CubicWeb development effort. Here are the decisions that were taken on Feb 1st, 2013.

Version 3.17

This version should be published before the end of March and will finish all the things that are work in progress. It will include:

  • the refactoring necessary to introduce persistant sessions,
  • the shrinking of web/views: everything that does not deserve its own cube (like sioc, embed, geocoding, etc) will go into a cube named legacyui (this will open the door to squareui),
  • stop serving pages with "content-type: application/xhtml",
  • handling postgresql schemas (will require a new version of logilab.database),
  • a new logo.

Squareui

Once the cube legacyui extracted (in version 3.17), it will be possible to move forward swiftly with squareui. Due to its other duties, one can not expect the core CW team to develop squareui. People interested will be in charge and ideally the squareui cube could be released when cubicweb 3.17 will be published.

Cleaning up the backlog

The lead CW developers will spend about 20% of their time cleaning up the ticket backlog at the forge (900 open tickets and 50 in progress !)

The first step will be to reduce the number of tickets "in progress", then to organize the open tickets and merge the duplicates.

Version 3.18

This version is due at the end of may 2013. It will include:

  • persisting sessions,
  • WSGI,
  • RESTfulness: support for HTTP verbs PUT / DELETE, enforcement of the semantics of GET / POST (may be difficult to maintain backward-compatibility)

Mid-term goals

The mid-term goals are:

  • possibility to add new base types (Array, HStore, Geometry, TSVector, etc.) that would use extensions from the SQL backend

  • FROM clause in rql queries

  • websockets

  • defining attribute on relations and defining "virtual" relations or rules:

    class Contribution(EntityType):
        author = SubjectRelation('Person', cardinality='1*', inlined=True)
        book = SubjectRelation('Book', cardinality='1*', inlined=True)
        role = SubjectRelation('Role', cardinality='1*', inlined=True)
    
    preface_writer = VirtualRelation('C is Contribution, C author S, C book O, '
                                     'C role R, R name "preface writer"')
    

    And:

    Any P WHERE B is Book, P preface_writer B
    

    Will we need a materialized view in the database, a standard relation maintained by hooks, rewrite the RQL on-the-fly ? Time will tell.

  • cards with logic (mustache js templates for example)

  • coffeescript ? brython ? javascript ? prototype something with CubicDB + WebService that outputs json + user interface in full javascript

  • package separately Cubic(Web)DB et CubicWeb ?

  • think about the overall architecture (using WSGI, persistent sessions, etc.), and find solutions that fit a distributed architecture (look at paste.deploy, circus, etc.)

  • clean up the javascript en web/data/*.js

  • configurable metadata, managing the size of the entities table

  • more SPARQL

  • namespaces for the data models of the cubes

As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.

Logilab's next roadmap meeting will be held at the beginning of April 2013.


What's new in CubicWeb 3.16

2013/01/23 by Aurelien Campeas

What's new in CubicWeb 3.16?

New functionalities

  • Add a new dataimport store (SQLGenObjectStore). This store enables a fast import of data (entity creation, link creation) in CubicWeb, by directly flushing information in SQL. This may only be used with PostgreSQL, as it requires the 'COPY FROM' command.

API changes

  • Orm: set_attributes and set_relations are unified (and deprecated) in favor of cw_set that works in all cases.

  • db-api/configuration: all the external repository connection information is now in an URL (see #2521848), allowing to drop specific options of pyro nameserver host, group, etc and fix broken ZMQ source. Configuration related changes:

    • Dropped 'pyro-ns-host', 'pyro-instance-id', 'pyro-ns-group' from the client side configuration, in favor of 'repository-uri'. NO MIGRATION IS DONE, supposing there is no web-only configuration in the wild.
    • Stop discovering the connection method through repo_method class attribute of the configuration, varying according to the configuration class. This is a first step on the way to a simpler configuration handling.

    DB-API related changes:

    • Stop indicating the connection method using ConnectionProperties.
    • Drop _cnxtype attribute from Connection and cnxtype from Session. The former is replaced by a is_repo_in_memory property and the later is totaly useless.
    • Turn repo_connect into _repo_connect to mark it as a private function.
    • Deprecate in_memory_cnx which becomes useless, use _repo_connect instead if necessary.
  • the "tcp://" uri scheme used for ZMQ communications (in a way reminiscent of Pyro) is now named "zmqpickle-tcp://", so as to make room for future zmq-based lightweight communications (without python objects pickling).

  • Request.base_url gets a secure=True optional parameter that yields an https url if possible, allowing hook-generated content to send secure urls (e.g. when sending mail notifications)

  • Dataimport ucsvreader gets a new boolean ignore_errors parameter.

Unintrusive API changes

  • Drop of cubicweb.web.uicfg.AutoformSectionRelationTags.bw_tag_map, deprecated since 3.6.

User interface changes

  • The RQL search bar has now some auto-completion support. It means relation types or entity types can be suggested while typing. It is an awesome improvement over the current behaviour !
  • The action box associated with table views (from tableview.py) has been transformed into a nice-looking series of small tabs; it means that the possible actions are immediately visible and need not be discovered by clicking on an almost invisible icon on the upper right.
  • The uicfg module has moved to web/views/ and ui configuration objects are now selectable. This will reduce the amount of subclassing and whole methods replacement usually needed to customize the ui behaviour in many cases.
  • Remove changelog view, as neither cubicweb nor known cubes/applications were properly feeding related files.

Other changes

  • 'pyrorql' sources will be automatically updated to use an URL to locate the source rather than configuration option. 'zmqrql' sources were broken before this change, so no upgrade is needed...
  • Debugging filters for Hooks and Operations have been added.
  • Some cubicweb-ctl commands used to show the output of msgcat and msgfmt; they don't anymore.