There has been a growing interest in ZMQ in the past months, due to its ability to efficiently deal with message passing, while being light and robust.
We have worked on introducing ZMQ in the CubicWeb framework for various uses :
- As a replacement/alternative to the Pyro source, that is used to connect to distant instances. ZMQ may be used as a lighter and more efficient alternative to Pyro. The main idea here is to use the send_pyobj/recv_pyobj API of PyZMQ (python wrapper of ZMQ) to execute methods on the distant Repository in a totally transparent way for CubicWeb.
- As a JSONServer. Indeed, ZMQ could be used to share data between a server and any requests done through ZMQ. The request is just a string of RQL, and the response is the result set formatted in Json.
- As the building block for a simple notification (publish/subscribe) system between CubicWeb instances. A component can register its interest in a particular topic, and receive a callback whenever a corresponding message is received. At this point, this mechanism is used in CubicWeb to notify other instances that they should invalidate their caches when an entity is deleted.
What's new in CubicWeb 3.16?
- Add a new dataimport store (SQLGenObjectStore). This store enables a fast
import of data (entity creation, link creation) in CubicWeb, by directly
flushing information in SQL. This may only be used with PostgreSQL, as it
requires the 'COPY FROM' command.
Orm: set_attributes and set_relations are unified (and
deprecated) in favor of cw_set that works in all cases.
db-api/configuration: all the external repository connection information is
now in an URL (see #2521848),
allowing to drop specific options of pyro nameserver host, group, etc and fix
broken ZMQ source. Configuration related changes:
- Dropped 'pyro-ns-host', 'pyro-instance-id', 'pyro-ns-group' from the client side
configuration, in favor of 'repository-uri'. NO MIGRATION IS DONE,
supposing there is no web-only configuration in the wild.
- Stop discovering the connection method through repo_method class attribute
of the configuration, varying according to the configuration class. This is
a first step on the way to a simpler configuration handling.
DB-API related changes:
- Stop indicating the connection method using ConnectionProperties.
- Drop _cnxtype attribute from Connection and cnxtype from
Session. The former is replaced by a is_repo_in_memory property
and the later is totaly useless.
- Turn repo_connect into _repo_connect to mark it as a private function.
- Deprecate in_memory_cnx which becomes useless, use _repo_connect instead
the "tcp://" uri scheme used for ZMQ
communications (in a way reminiscent of Pyro) is now named
"zmqpickle-tcp://", so as to make room for future zmq-based lightweight
communications (without python objects pickling).
Request.base_url gets a secure=True optional parameter that yields
an https url if possible, allowing hook-generated content to send
secure urls (e.g. when sending mail notifications)
Dataimport ucsvreader gets a new boolean ignore_errors
- Drop of cubicweb.web.uicfg.AutoformSectionRelationTags.bw_tag_map,
deprecated since 3.6.
- The RQL search bar has now some auto-completion support. It means
relation types or entity types can be suggested while typing. It is
an awesome improvement over the current behaviour !
- The action box associated with table views (from tableview.py)
has been transformed into a nice-looking series of small tabs; it
means that the possible actions are immediately visible and need not
be discovered by clicking on an almost invisible icon on the upper
- The uicfg module has moved to web/views/ and ui configuration
objects are now selectable. This will reduce the amount of
subclassing and whole methods replacement usually needed to
customize the ui behaviour in many cases.
- Remove changelog view, as neither cubicweb nor known cubes/applications
were properly feeding related files.
- 'pyrorql' sources will be automatically updated to use an URL to locate the source
rather than configuration option. 'zmqrql' sources were broken before this change,
so no upgrade is needed...
- Debugging filters for Hooks and Operations have been added.
- Some cubicweb-ctl commands used to show the output of msgcat and
msgfmt; they don't anymore.