Blog entries

  • CubicWeb roadmap meeting on November 6th, 2014

    2014/11/03 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in September 2014.

    Here is the report about the November 6th, 2014 meeting. Christophe de Vienne (Unlish) joined us to express their concerns and discuss the future of CubicWeb. Dimitri Papadopoulos (CEA) could not come.

    Versions

    Version 3.17

    This version is stable but old and maintainance will continue only as long as some customers will be willing to pay for it (current is 3.17.17).

    If you're still using 3.17, you should go directly to 3.19.

    Version 3.18

    This version is stable but old and maintained (current is 3.18.6).

    Version 3.19

    This version is stable and maintained (current is 3.19.5).

    Version 3.20

    This version is still under development but should be released very soon now (expected next week). Its main feature being the inclusion of CWEP-002 (computed attributes and relations), along with many small improvement patches.

    For details read list of tickets for CubicWeb 3.20.0.

    We would have loved to integrate the pyramid cube in this release, but the debian packaging effort needed by the pyramid stack is quite big and is acceptable if we target jessie only (at decent price).

    Version 3.21

    For now, the roadmap for 3.21 is still the complete removal of the dbapi, the merging of Connection and ClientConnection, and possibly including CWEP-003 (adding a FROM clause to RQL).

    Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported.

    Integration of CWEP-004 is being discussed.

    Version 4.0

    We expect to accelerate development of CubicWeb 4, which exact roadmap is still to be discussed, but we may already want:

    • be pyramid-based (remove twisted, auth management, etc.),
    • do not have anything left of old dbapi and ClientConnection,
    • integrate squareui as main (and only) web-ui "template" or remove web generation (almost) completely from cubicweb-core and provide it only through the cube system.

    CWEPs

    Here is the status of open CubicWeb Evolution Proposals:

    to be written

    Work in progress

    Some work is in progress around CKAN, DCAT and othr Open Data and Semantic Web related technologies.

    Agenda

    Next roadmap meeting will be held at the beginning of january 2015 at Logilab, and Christophe and Dimitri (or Yann) are invited.

    Open Discussions

    Migration:

    • AppObjects should not be loaded by default
    • Have a look at Alembic the migration tool for SQLAlchemy and take inspiration from there

  • Logilab's roadmap for CubicWeb on March 7th, 2014

    2014/03/10 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. Here is the report about the Mar 7th, 2014 meeting. The previous report posted to the blog was the january 2014 roadmap.

    Version 3.17

    This version is stable but old and maintainance will stop in a few weeks (current is 3.17.13 and 3.17.14 is upcoming).

    Version 3.18

    This version is stable and maintained (current is 3.18.3 and 3.18.4 is upcoming).

    Version 3.19

    This version is about to be published. It includes a heavy refactoring that modifies sessions and sources to lay the path for CubicWeb 4.

    For details read list of tickets for CubicWeb 3.19.0.

    Version 3.20

    This version will try to reduce as much as possible the stock of patches in the state "reviewed", "awaiting review" and "in progress". If you have had something in the works that has not been accepted yet, please ready it for 3.20 and get it merged.

    It should also include the work done for CWEP-002 (computed attributes and relations) and CWEP-003 (adding a FROM clause to RQL).

    For details read list of tickets for CubicWeb 3.20.0.

    Cubes

    Here is a list of cubes that had versions published over the past two months: addressbook, awstats, blog, bootstrap, brainomics, comment, container, dataio, genomics, invoice, mediaplayer, medicalexp, neuroimaginge, person, preview, questionnaire, securityprofile, simplefacet, squareui, tag, tracker, varnish, vcwiki, vtimeline.

    Here are a the new cubes we are pleased to announce:

    collaboration is a building block that reuses container and helps to define collaborative workflows where entities are cloned, modified and shared.

    Our priorities for the next two months are collaboration and container, then narval/apycot, then mercurial-server, then rqlcontroller and signedrequest, then imagesearch.

    Mid-term goals

    The work done for CWEP-0002 (computed attributes and relations) is expected to land in CubicWeb 3.20.

    The work done for CWEP-0003 (explicit data source federation using FROM in RQL) is expected to land in CubicWeb 3.20.

    Tools to diagnose performance issues would be very useful. Maybe in 3.21 ?

    Caching session data would help and some work was done on this topic during the sprint in february. Maybe in 3.22 ?

    WSGI has made progress lately, but still needs work. Maybe in 3.23 ?

    RESTfulness is a goal. Maybe in 3.24 ?

    Maybe 3.25 will be in fact 4.0 ?

    Events

    A spring sprint will take place in Logilab's offices in Paris from April 28th to 30th. We invite all the interested parties to join us there!

    Last but not least

    As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.

    Logilab's next roadmap meeting will be held at the beginning of may 2014.


  • Logilab's roadmap for CubicWeb on May 15th, 2014

    2014/05/21 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. Here is the report about the May 15th, 2014 meeting. The previous report posted to the blog was the march 2014 roadmap.

    Versions

    Version 3.17

    This version is stable but old and maintainance will continue only as long as some customers will be willing to pay for it (current is 3.17.15).

    Version 3.18

    This version is stable and maintained (current is 3.18.4).

    Version 3.19

    This version was published at the end of April. It includes support for Cross Origin Resource Sharing (CORS) and a heavy refactoring that modifies sessions and sources to lay the path for CubicWeb 4.

    For details read the release notes or the list of tickets for CubicWeb 3.19.0.

    Version 3.20

    This version is under development. It will try to reduce as much as possible the stock of patches in the state "reviewed", "awaiting review" and "in progress". If you have had something in the works that has not been accepted yet, please ready it for 3.20 and get it merged.

    It should also include the work done for CWEP-002 (computed attributes and relations) and the merging of Connection and ClientConnection if it happens to be simple enough to get done quickly (in case the removal of dbapi would really help, this merging will wait for 3.21).

    For details read list of tickets for CubicWeb 3.20.0.

    Version 3.21 (or maybe 4.0?)

    Removal of the dbapi and merging of CWEP-003 (adding a FROM clause to RQL).

    Cubes

    Here is a list of cubes that had versions published over the past two months: accidents, awstats, book, bootstrap, brainomics, cmt, collaboration, condor, container, dataio, expense, faq, file, forge, forum, genomics, geocoding, inlineedit, inventory, keyword, link, mailinglist, mediaplayer, medicalexp, nazcaui, ner, neuroimaging, newsaggregator, processing, questionnaire, rqlcontroller, semnews, signedrequest, squareui, task, testcard, timesheet, tracker, treeview, vcsfile, workorder.

    Here are a the new cubes we are pleased to announce:

    rqlcontroller receives via a POST a list of RQL queries and executes them. This is a way to build web services.

    wsme is helping build a web service API on top of a CubicWeb database.

    signedrequest is a simple token based authentication system. This is a way for scripts or callback urls to access an instance without login/pwd information.

    relationwidget is a widget usable in forms to edit relationships between objects. It depends on CubicWeb 3.19.

    searchui is an experiment on adding blocks to the list of facets that allow building complex RQL queries step by step by clicking with the mouse instead of directly writing the RQL with the keyboard.

    ckan is using the REST API of a CKAN data portal to mirror its content.

    CWEPs

    Here is the status of open CubicWeb Evolution Proposals:

    CWEP-0002 is now in good shape and the goal is to have it merged into 3.20. It lacks some documentation and a migration script.

    CWEP-0003 has made good progress during the latest sprint, but will need a thorough review before being merged. It will probably not be ready for 3.20 and have to wait for 3.21.

    New CWEPs are expected to be written for clarifying the API of the _cw object, supporting persistent sessions and improving the performance of massive imports.

    Visual identity

    CubicWeb has a new logo that will appear before the end of may on its revamped homepage at http://www.cubicweb.org

    Last but not least

    As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.

    Logilab's next roadmap meeting will be held at the beginning of july 2014.


  • Logilab's roadmap for CubicWeb on September 6th, 2013

    2013/09/17 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. Here is the report about the Sept 6th, 2013 meeting. The previous report posted to the blog was the february 2013 roadmap.

    Version 3.17

    This version is now stable and maintained (release 3.17.7 is upcoming). It added a couple features and focused on putting CW to the diet by extracting some functionnalities provided by the core into external cubes: sioc, embed, massmailing, geocoding, etc.

    For details read what's new in CubicWeb 3.17.

    Version 3.18

    This version is now freezed and will be published as soon as all the patches are tested and merged. Since we have a lot of work for clients until the end of the year at Logilab, the community should feel free to help (as usual) if it wants this version to be released rather sooner than later.

    This version will remove the ldapuser source that is replaced by ldapfeed, implement Cross Origin Resource Sharing, drop some very old compatibility code, deprecate the old version of the multi-source system and provide various other features and bugfixes.

    For details read list of tickets for CubicWeb 3.18.0.

    Version 3.19

    This version will probably be publish early next year (read january or february 2014) unless someone who is not working at Logilab takes responsibility for its release.

    It should include the heavy refactoring work done by Pierre-Yves and Sylvain over the past year, that modifies sessions and sources to lay the path for CubicWeb 4.

    For details read list of tickets for CubicWeb 3.19.0 or take a look at this head.

    Squareui

    Since Orbui changes the organization of the default user interface on screen, it was decided to share the low-level bootstrap related views that could be shared and build a SquareUI cube that would conform design choices of the default UI.

    Logilab is now developping all its new projects based on Squareui 0.2. Read about it on the mailing list archives.

    Mid-term goals

    The mid-term goals include better REST support (Representational State Transfer), complete WSGI (Python's Web Server Gateway Interface) and the FROM clause for RQL queries (to reinvent db federation outside of the core).

    Cubes

    Our current plan is to extract as much as possible to cubes. We started CubicWeb many years ago with the Python motto "batteries included", but have since realized that having too much in the core contributes to making CubicWeb difficult to learn.

    Since we would very much like the community to grow, we are now aiming for something more balanced, like Mercurial does. The core is designed such that most features can be developed as an extension. Once they are stable, popular extensions can be moved to the main library that is distributed with the core, and be activated with a switch in the configuration file.

    Several cubes are under active development: oauth, signedrequest, dataio, etc.

    Last but not least

    As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.

    Logilab's next roadmap meeting will be held at the beginning of November 2013.


  • Logilab's roadmap for CubicWeb as of February 2013

    2013/02/04 by Nicolas Chauvat

    The Logilab team now holds a roadmap meeting every two months to plan its CubicWeb development effort. Here are the decisions that were taken on Feb 1st, 2013.

    Version 3.17

    This version should be published before the end of March and will finish all the things that are work in progress. It will include:

    • the refactoring necessary to introduce persistant sessions,
    • the shrinking of web/views: everything that does not deserve its own cube (like sioc, embed, geocoding, etc) will go into a cube named legacyui (this will open the door to squareui),
    • stop serving pages with "content-type: application/xhtml",
    • handling postgresql schemas (will require a new version of logilab.database),
    • a new logo.

    Squareui

    Once the cube legacyui extracted (in version 3.17), it will be possible to move forward swiftly with squareui. Due to its other duties, one can not expect the core CW team to develop squareui. People interested will be in charge and ideally the squareui cube could be released when cubicweb 3.17 will be published.

    Cleaning up the backlog

    The lead CW developers will spend about 20% of their time cleaning up the ticket backlog at the forge (900 open tickets and 50 in progress !)

    The first step will be to reduce the number of tickets "in progress", then to organize the open tickets and merge the duplicates.

    Version 3.18

    This version is due at the end of may 2013. It will include:

    • persisting sessions,
    • WSGI,
    • RESTfulness: support for HTTP verbs PUT / DELETE, enforcement of the semantics of GET / POST (may be difficult to maintain backward-compatibility)

    Mid-term goals

    The mid-term goals are:

    • possibility to add new base types (Array, HStore, Geometry, TSVector, etc.) that would use extensions from the SQL backend

    • FROM clause in rql queries

    • websockets

    • defining attribute on relations and defining "virtual" relations or rules:

      class Contribution(EntityType):
          author = SubjectRelation('Person', cardinality='1*', inlined=True)
          book = SubjectRelation('Book', cardinality='1*', inlined=True)
          role = SubjectRelation('Role', cardinality='1*', inlined=True)
      
      preface_writer = VirtualRelation('C is Contribution, C author S, C book O, '
                                       'C role R, R name "preface writer"')
      

      And:

      Any P WHERE B is Book, P preface_writer B
      

      Will we need a materialized view in the database, a standard relation maintained by hooks, rewrite the RQL on-the-fly ? Time will tell.

    • cards with logic (mustache js templates for example)

    • coffeescript ? brython ? javascript ? prototype something with CubicDB + WebService that outputs json + user interface in full javascript

    • package separately Cubic(Web)DB et CubicWeb ?

    • think about the overall architecture (using WSGI, persistent sessions, etc.), and find solutions that fit a distributed architecture (look at paste.deploy, circus, etc.)

    • clean up the javascript en web/data/*.js

    • configurable metadata, managing the size of the entities table

    • more SPARQL

    • namespaces for the data models of the cubes

    As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.

    Logilab's next roadmap meeting will be held at the beginning of April 2013.


  • Thoughts on CubicWeb 4.0

    2012/05/14 by Sylvain Thenault

    This is a fairly technical post talking about the structural changes I would like to see in CubicWeb's near future. Let's call that CubicWeb 4.0! It also drafts ideas on how to go from here to there. Draft, really. But that will eventually turn into a nice roadmap hopefully.

    The great simplification

    Some parts of cubicweb are sometimes too hairy for different reasons (some good, most bad). This participates in the difficulty to get started quickly. The goal of CubicWeb 4.0 should be to make things simpler :

    • Fix some bad old design.
    • Stop reinventing the wheel and use widely used libraries in the Python Web World. This extends to benefitting from state of the art libraries to build nice and flexible UI such as Bootstrap, on top of the JQuery foundations (which could become as prominent as the Python standard library in CubicWeb, the development team should get ready for it).
    • If there is a best way to do something, just do it and refrain from providing configurability and options.

    On the road to Bootstrap

    First, a few simple things could be done to simplify the UI code:

    • drop xhtml support: always return text/html content type, stop bothering with this stillborn stuff and use html5
    • move away everything that should not be in the framework: calendar?, embedding, igeocodable, isioc, massmailing, owl?, rdf?, timeline, timetable?, treeview?, vcard, wdoc?, xbel, xmlrss?

    Then we should probably move the default UI into some cubes (i.e. the content of cw.web.views and cw.web.data). Besides making the move to Bootstrap easier, this should also have the benefit of making clearer that this is the default way to build an (automatic) UI in CubicWeb, but one may use other, more usual, strategies (such as using a template language).

    At a first glance, we should start with the following core cubes:

    • corelayout, the default interface layout and generic components. Modules to backport there: application (not an appobject yet), basetemplates, error, boxes, basecomponents, facets, ibreadcrumbs, navigation, undohistory.
    • coreviews, the default generic views and forms. Modules to backport there: actions, ajaxedit, baseviews, autoform, dotgraphview, editcontroller, editforms, editviews, forms, formrenderers, primary, json, pyviews, tableview, reledit, tabs.
    • corebackoffice, the concrete views for the default back-office that let you handle users, sources, debugging, etc. through the web. Modules to backport here: cwuser, debug, bookmark, cwproperties, cwsources, emailaddress, management, schema, startup, workflow.
    • coreservices, the various services, not directly related to display of something. Modules to backport here: ajaxcontroller, apacherewrite, authentication, basecontrollers, csvexport, idownloadable, magicsearch, sessions, sparql, sessions, staticcontrollers, urlpublishing, urlrewrite.

    This is a first draft that will need some adjustements. Some of the listed modules should be split (e.g. actions, boxes,) and their content moved to different core cubes. Also some modules in cubicweb.web packages may be moved to the relevant cube.

    Each cube should provide an interface so that one could replace it with another one. For instance, move from the default coreviews and corelayout cube to bootstrap based ones. This should allow a nice migration path from the current UI to a Bootstrap based UI. Bootstrap should probably be introduced bottom-up: start using it for tables, lists, etc. then go up until the layout defined in the main template. The Orbui experience should greatly help us by pointing at hot spots that will have to be tackled, as well as by providing a nice code base from which we should start.

    Regarding current implementation, we should take care that Contextual components are a powerful way to build "pluggable" UI, but we should probably add an intermediate layer that would make more obvious / explicit:

    • what the available components are
    • what the available slots are
    • which component should go in which slot when possible

    Also at some point, we should take care to separate view's logic from HTML generation: our experience with client works shows that a common need is to use the logic but produce a different HTML. Though we should wait for more use of Bootstrap and related HTML simplification to see if the CSS power doesn't somewhat fulfill that need.

    On the road to proper tasks management

    The current looping task / repo thread mecanism is used for various sort of things and has several problems:

    • tasks don't behave similarly in a multi-instances configuration (some should be executed in a single instance, some in a subset); the tasks system has been originally written in a single instance context; as of today this is (sometimes) handled using configuration options (that will have to be properly set in each instance configuration file);
    • tasks is a repository only api but we also need web-side tasks;
    • there is probably some abuse of the system that may lead to unnecessary resources usage.

    Analyzing a sample http://www.logilab.org/ instance, below are the running looping task by categories. Tasks that have to run on each web instance:

    • clean_sessions, automatically closes unused repository sessions. Notice cw.etwist.server also records a twisted task to clean web sessions. Some changes are imminent on this, they will be addressed in the upcoming refactoring session (that will become more and more necessary to move on several points listed here).
    • regular_preview_dir_cleanup (preview cube), cleanup files in the preview filesystem directory. Could be executed by a (some of the) web instance(s) provided that the preview directory is shared.

    Tasks that should run on a single instance:

    • update_feeds, update copy based sources (e.g. datafeed, ldapfeed). Controlled by 'synchronize' source configuration (persistent source attribute that may be overridden by instance using CWSourceHostConfig entities)
    • expire_dataimports, delete CWDataImport entities older than an amount of time specified in the 'logs-lifetime' configuration option. Not controlled yet.
    • cleanup_auth_cookies (rememberme cube), delete CWAuthCookie entities whose life-time is exhausted. Not controlled yet.
    • cleaning_revocation_key (forgotpwd cube), delete Fpasswd entities with past revocation_date. Not controlled yet.
    • cleanup_plans (narval cube), delete Plan entities instance older than an amount of time specified in the configuration. If 'plan-cleanup-delay' is set to an empty value, the task isn't started.
    • refresh_local_repo_caches (vcsfile cube), pull or clone vcs repositories cache if the Repository entity ask to import_revision_content (hence web instance should have up to date cache to display files content) or if 'repository-import' configuration option is set to 'yes'; import vcs repository content as entities if 'repository-import' configuration option and it is coming from the system source.

    Some deeper thinking is needed here so we can improve things. That includes thinking about:

    • the inter-instances messages bus based on zmq and introduced in 3.15,
    • the Celery project (http://celeryproject.org/), an asynchronous task queue, widely used and written in Python,

    Remember the more cw independent the tasks are, the better it is. Though we still want an 'all-integrated' approach, e.g. not relying on external configuration of Unix specific tools such as CRON. Also we should see if a hard-dependency on Celery or a similar tool could be avoided, and if not if it should be considered as a problem (for devops).

    On the road to an easier configuration

    First, we should drop the different behaviour according to presence of a '.hg' in cubicweb's directory. It currently changes the location where cubicweb external resources (js, css, images, gettext catalogs) are searched for. Speaking of implementation:

    • shared_dir returns the cubicweb.web package path instead of the path to the shared cube,
    • i18n_lib_dir returns the cubicweb/i18n directory path instead of the path to the shared/i18n cube,
    • migration_scripts_dir returns the cubicweb/misc/migration directory path instead of share/cubicweb/migration.

    Moving web related objects as proposed in the Bootstrap section would resolve the problem for the content web/data and most of i18n (though some messages will remain and additional efforts will be needed here). By going further this way, we may also clean up some schema code by moving cubicweb/schemas and cubicweb/misc/migration to a cube (though only a small benefit is to be expected here).

    We should also have fewer environment variables... Let's see what we have today:

    • CW_INSTANCES_DIR, where to look for instances configuration
    • CW_INSTANCES_DATA_DIR, where to look for instances persistent data files
    • CW_RUNTIME_DIR, where to look for instances run-time data files
    • CW_MODE, set to 'system' or 'user' will predefine above environment variables differently
    • CW_CUBES_PATH, additional directories where to look for cubes
    • CW_CUBES_DIR, location of the system 'cubes' directory
    • CW_INSTALL_PREFIX, installation prefix, from which we can compute path to 'etc', 'var', 'share', etc.

    I would propose the following changes:

    • CW_INSTANCES_DIR is turned into CW_INSTANCES_PATH, and defaults to ~/etc/cubicweb.d if it exists and /etc/cubicweb.d (on Unix platforms) otherwise;
    • CW_INSTANCES_DATA_DIR and CW_RUNTIME_DIR are replaced by configuration file options, with smart values generated at instance creation time;
    • the above change should make CW_MODE useless;
    • CW_CUBES_DIR is to be dropped, CW_CUBES_PATH should be enough;
    • regarding CW_INSTALL_PREFIX, I'm lacking experience with non-hg-or-debian installations and don't know if this can be avoided or not.

    Last but not least (for the moment), the 'web' / 'repo' / 'all-in-one' configurations, and the fact that the associated configuration file changes stinks. Ideas to stop doing this:

    • one configuration file per instance, with all options provided by installed parts of the framework used by the application.
    • activate 'services' (or not): web server, repository, zmq server, pyro server. Default services to be started are stored in the configuration file.

    There is probably more that can be done here (less configuration options?), but that would already be a great step forward.

    On the road to...

    The following projects should be investigated to see if we could benefit from them:

    Discussion

    Remember the following goals: migration of legacy code should go smoothly. In a perfect world every application should be able to run with CubicWeb 4.0 until the backwards compatibility code is removed (and CubicWeb 4.0 will probably be released as 4.0 at that time).

    Please provide feedbacks:

    • do you think choices proposed above are good/bad choices? Why?
    • do you know some additional libraries that should be investigated?
    • do you have other changes in mind that could/should be done in cw 4.0?

  • Logilab's roadmap for CubicWeb on November 8th, 2013

    2013/11/11 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. Here is the report about the Nov 8th, 2013 meeting. The previous report posted to the blog was the september 2013 roadmap.

    Version 3.17

    This version is stable and maintained (cubicweb 3.17.11 is upcoming).

    Version 3.18

    This version was supposed to be released in september or october, but is stalled at the integration stage. All open tickets were moved to 3.19 and existing patches that are not ready to be merged will be more aggressively delayed to 3.19. The goal is to release 3.18 as soon as possible.

    For details read list of tickets for CubicWeb 3.18.0.

    Version 3.19

    This version will probably be published early next year (read january or february 2014). it is planned to include a heavy refactoring that modifies sessions and sources to lay the path for CubicWeb 4.

    For details read list of tickets for CubicWeb 3.19.0.

    Squareui

    Logilab is now developping all its new projects based on Squareui (and Bootstrap 3.0). Squareui can be considered as a usable beta, but not as feature-complete.

    Logilab is looking for a UX designer to work on the general ergonomy of CubicWeb. Read the job offer.

    Mid-term goals

    The mid-term goals include better REST support (Representational State Transfer), complete WSGI (Python's Web Server Gateway Interface) and the FROM clause for RQL queries (to reinvent db federation outside of the core).

    On the front-end side, it would be nice to be able to improve forms, maybe with client-side javascript and better support for a "json on server, js in browser" separation of concerns.

    Cubes

    A cube oauth was contributed in large part by Unlish, a startup that is using CubicWeb to implement its service.

    A cube vcwiki is being developed by Logilab, to manage the content of a wiki with a version control system (built with the cube vcsfile).

    Last but not least

    As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.

    Logilab's next roadmap meeting will be held at the beginning of january 2014.


  • Logilab's roadmap for CubicWeb on January 9th, 2014

    2014/01/14 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. Here is the report about the Jan 9th, 2014 meeting. The previous report posted to the blog was the november 2013 roadmap.

    Version 3.17

    This version is stable and maintained (current is 3.17.11 and 3.17.12 is upcoming).

    Version 3.18

    This version was released on Jan 10th. Read the release notes or the details of CubicWeb 3.18.0.

    Version 3.19

    This version includes a heavy refactoring that modifies sessions and sources to lay the path for CubicWeb 4. It is currently the default development head in the repository and is expected to be released before the end of january.

    For details read list of tickets for CubicWeb 3.19.0.

    Version 3.20

    This version will try to reduce as much as possible the stock of patches in the state "reviewed", "awaiting review" and "in progress". If you have had something in the works that has not been accepted yet, please ready it for 3.20 and get it merged.

    For details read list of tickets for CubicWeb 3.20.0.

    Cubes

    The current trend is to develop more and more new features in dedicated cubes than to add more code to the core of CubicWeb. If you thought CubicWeb development was slowing down, you made a mistake, because cubes are ramping up.

    Here is a list of versions that were published in the past two months: timesheet, postgis, leaflet, bootstrap, worker, container, embed, geocoding, vcreview, trackervcs, vcsfile, zone, dataio, mercurial-server, queueing, questionnaire, genomics, medicalexp, neuroimaging, brainomics, elections.

    Here are a the new cubes we are pleased to announce:

    Bootstrap works and we do not create a new application without it.

    relationwidget provides a modal window to edit relations in forms (use uicfg to activate it).

    resourcepicker provides a modal window to insert links to images and files into structured text.

    rqlcontroller allows to use the INSERT, DELETE and SET keywords when sending RQL queries over HTTP. It returns JSON. Get used to it and you may forget about asking for specific web services in your apps, for it is a generic web service.

    imagesearch is an image gallery with facets. You may use it as a demo of a visual search tool.

    Mid-term goals

    A new repository was created to have all the CubicWeb Evolution Proposals in one place.

    CWEP-0002 is a work in progress about computed relations and computed attributes, or maybe more. It will be a focus of the next sprint and is targeted at CubicWeb 3.20.

    A new CWEP is expected about the adding FROM keyword to RQL to implement explicit data source federation. It will be a focus of the next sprint and is targeted at CubicWeb 3.21.

    Tools to diagnose performance issues would be very useful. Maybe in 3.22 ?

    Caching session data would help. Maybe in 3.23 ?

    WSGI has made progress lately, but still needs work. Maybe in 3.24 ?

    RESTfulness is a goal. Maybe in 3.25 ?

    Maybe 3.26 will be in fact 4.0 ?

    Events

    A sprint will take place in Logilab's offices in Paris around mid-february or at the end of april. We invite all the interested parties to join us there!

    Last but not least

    As already said on the mailing list, other developers and contributors are more than welcome to share their own goals in order to define a roadmap that best fits everyone's needs.

    Logilab's next roadmap meeting will be held at the beginning of march 2014.


  • CubicWeb roadmap meeting on September 4th, 2014

    2014/09/01 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in July 2014.

    Here is the report about the September 4th, 2014 meeting. Christophe de Vienne (Unlish) and Dimitri Papadopoulos (CEA) joined us to express their concerns and discuss the future of CubicWeb.

    Versions

    Version 3.17

    This version is stable but old and maintainance will continue only as long as some customers will be willing to pay for it (current is 3.17.16 with 3.17.17 in development).

    Version 3.18

    This version is stable and maintained (current is 3.18.5 with 3.18.6 in development).

    Version 3.19

    This version is stable and maintained (current is 3.19.3 with 3.19.4 in development).

    Version 3.20

    This version is under development. It will try to reduce as much as possible the stock of patches in the state "reviewed", "awaiting review" and "in progress". If you have had something in the works that has not been accepted yet, please ready it for 3.20 and get it merged.

    It should still include the work done for CWEP-002 (computed attributes and relations).

    For details read list of tickets for CubicWeb 3.20.0.

    Version 3.21

    Removal of the dbapi, merging of Connection and ClientConnection, CWEP-003 (adding a FROM clause to RQL).

    Version 4.0

    When the work done for Pyramid will have been tested, it will become the default runner and a lot of things will be dropped: twisted, dead code, ui and core code that would be better cast into cubes, etc.

    This version could happen early in 2015.

    Cubes

    New cubes and libraries

    CWEPs

    Here is the status of open CubicWeb Evolution Proposals:

    CWEP-0002 full-featured implementation, to be merged in 3.20

    CWEP-0003 patches sent to the review. . Champion will be adim.

    Work in progress

    PyConFR

    Christophe will try to present at PyConFR the work he did on getting CubicWeb to work with Pyramid.

    Pip-friendly source layout

    Logilab and Christophe will try to make CubicWeb more pip/virtualenv-friendly. This may involve changing the source layout to include a sub-directory, but the impact on existing devs is expected to be too much and could be delayed to CubicWeb 4.0.

    Pyramid

    Christophe has made good progress on getting CubicWeb to work with Pyramid and he intends to put it into production real soon now. There is a Pyramid extension named pyramid_cubicweb and a CubicWeb cube named cubicweb-pyramid. Both work with CubicWeb 3.19. Christophe demonstrated using the debug toolbar, authenticating users with Authomatic and starting multiple workers with uWSGI.

    Early adopters are now invited to jump in and help harden the code!

    Agenda

    Logilab's next roadmap meeting will be held at the beginning of november 2014 and Christophe and Dimitri were invited.


  • CubicWeb roadmap meeting on July 3rd, 2014

    2014/06/26 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in May 2014.

    Here is the report about the July 3rd, 2014 meeting. Christophe de Vienne (Unlish) and Dimitri Papadopoulos (CEA) joined us to express their concerns and discuss the future of CubicWeb.

    Versions

    Version 3.17

    This version is stable but old and maintainance will continue only as long as some customers will be willing to pay for it (current is 3.17.15 with 3.17.16 in development).

    Version 3.18

    This version is stable and maintained (current is 3.18.5 with 3.18.6 in development).

    Version 3.19

    This version was published at the end of April and has now been tested on our internal servers. It includes support for Cross Origin Resource Sharing (CORS) and a heavy refactoring that modifies sessions and sources to lay the path for CubicWeb 4.

    For details read the release notes or the list of tickets for CubicWeb 3.19.0. Current is 3.19.2

    Version 3.20

    This version is under development. It will try to reduce as much as possible the stock of patches in the state "reviewed", "awaiting review" and "in progress". If you have had something in the works that has not been accepted yet, please ready it for 3.20 and get it merged.

    It should still include the work done for CWEP-002 (computed attributes and relations.

    For details read list of tickets for CubicWeb 3.20.0.

    Version 3.21 (or maybe 4.0?)

    Removal of the dbapi, merging of Connection and ClientConnection, CWEP-003 (adding a FROM clause to RQL).

    Cubes

    Cubes published over the past two months

    New cubes

    • cubicweb-frbr: Cube providing a schema based on FRBR entities
    • cubicweb-clinipath
    • cubicweb-fastimport

    CWEPs

    Here is the status of open CubicWeb Evolution Proposals:

    CWEP-0002 only missing a bit of migration support, to be finished soon for inclusion in 3.20.

    CWEP-0003 has been reviewed and is waiting for a bit of reshaping that should occurs soon. It's targeted for 3.21.

    New CWEPs are expected to be written for clarifying the API of the _cw object, supporting persistent sessions and improving the performance of massive imports.

    Work in progress

    Design

    The new logo is now published in the 3.19 line. David showed us his experimentation that modernize a forge's ui with a bit of CSS. There is still a bit of pressure on the bootstrap side though, as it still rely on heavy monkey-patching in the cubicweb-bootstrap cube.

    Data import

    Also, Dimitry expressed is concerns with the lack of proper data import API. We should soon have some feedback from Aurelien's cubicweb-fastimport experimentation, which may be an answer to Dimitry's need. In the end, we somewhat agreed that there were different needs (eg massive-no-consistency import vs not-so-big-but-still-safe), that cubicweb.dataimport was an attempt to answer them all and then cubicweb-dataio and cubicweb-fastimport were more specific responses. In the end we may reasonably hope that an API will emerge.

    Removals

    On his way to persistent sessions, Aurélien made a huge progress toward silence of warnings in the 3.19 tests. dbapi has been removed, ClientConnection / Connection merged. We decided to take some time to think about the recurring task management as it is related to other tricky topics (application / instance configuration) and it's not directly related to persistent session.

    Rebasing on Pyramid

    Last but not least, Christophe demonstrated that CubicWeb could basically live with Pyramid. This experimentation will be pursued as it sounds very promising to get the good parts from the two framework.

    Agenda

    Logilab's next roadmap meeting will be held at the beginning of september 2014 and Christophe and Dimitri were invited.


  • CubicWeb roadmap meeting on January 8th, 2015

    2015/01/05 by Nicolas Chauvat

    The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in November 2014.

    Here is the report about the January 8th, 2015 meeting.

    Christophe de Vienne (Unlish) and Aurélien Campéas (self-employed) joined us to express their concerns and discuss the future of CubicWeb.

    Versions

    Version 3.18

    This version is stable but old and maintained (current is 3.18.7).

    Version 3.19

    This version is stable and maintained (current is 3.19.8).

    Version 3.20

    This version has been released a few days ago. It has not been deployed on production systems yet.

    Its main features are:

    • virtual relations: a new ComputedRelation class can be used in schema.py; its rule attribute is an RQL snippet that defines the new relation.

    • computed attributes: an attribute can now be defined with a formula argument (also an RQL snippet); it will be read-only, and updated automatically.

      Both of these features are described in CWEP-002, and the updated "Data model" chapter of the CubicWeb book.

    • cubicweb-ctl plugins can use the cubicweb.utils.admincnx function to get a Connection object from an instance name.

    • new 'tornado' wsgi backend

    • session cookies have the HttpOnly flag, so they're no longer exposed to javascript

    • rich text fields can be formatted as markdown

    • the edit controller detects concurrent editions, and raises a ValidationError if an entity was modified between form generation and submission

    • cubicweb can use a postgresql "schema" (namespace) for its tables

    • cubicweb-ctl configure can be used to set values of the admin user credentials in the sources configuration file

    For details read list of tickets for CubicWeb 3.20.0.

    We would have loved to integrate the pyramid cube in this release, but the debian packaging effort needed by the pyramid stack is quite big and is acceptable if we target jessie only (at decent price).

    Version 3.21

    For now, the roadmap for 3.21 is still the complete removal of the dbapi, the merging of Connection and ClientConnection.

    Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported.

    Integration of CWEP-003 (FROM clause for RQL) and CWEP-004 (cubes as standard python packages) is being discussed.

    Version 4.0

    We expect to accelerate development of CubicWeb 4, which exact roadmap is still to be discussed, but we may already want:

    • be pyramid-based (remove twisted, auth management, etc.),
    • do not have anything left of old dbapi and ClientConnection,
    • integrate squareui as main (and only) web-ui "template" or remove web generation (almost) completely from cubicweb-core and provide it only through the cube system.

    Agenda

    Next roadmap meeting will be held at the beginning of march 2015 at Logilab. Interested parties are invited to get in touch.

    Open Discussions

    Refactoring the documentation

    Christophe de Vienne suggested to completely revamp the documentation and intends to lead this effort.

    Training material

    Aurélien Campéas asks if Logilab would be willing to share its training material under a free license to help interested parties organize and sell trainings.

    Towards making squareui the default rendering engine for cubicweb

    We are expecting to be able to use squareui/bootstrap as "rendering engine" for our forge applications (like http://www.cubicweb.org and http://www.logilab.org) as soon as possible. However to achieve to goal, there are still too many "visual bugs", some of which may require a discussion.

    Among others:

    • put the ctxtoolbar component in the <nav> div
    • each box component should have an icon (what API for this?)
    • we cannot easily make the left column of the main template responsive-aware (requires to change the html flow), so it's probably best to take inspiration from things like http://wrapbootstrap.com/preview/WB0N89JMK
    • facet boxes are a mess, there is no simple solution to have a "smart layout"

    Migration

    • AppObjects should not be loaded by default
    • Have a look at Alembic the migration tool for SQLAlchemy and take inspiration from there.