cubicweb #5503548 rql2sql fails to translate a query [resolved]
Some queries fails with a KeyError in rql2sql. The following unittest reproduce the issue: def test_rql2sql_ll(self): query = """ Any X WHERE X is Event, EXISTS(X at_place A, A refpoint B, A is Place), X organized_by X_ORGANIZED_BY, X external_url X_EXTERNAL_URL? """ with self.admin_access.repo_cnx() as cnx: john_eid = cnx.find('CWUser', login='john').one().eid args = {'D': john_eid} with self.repo.internal_cnx() as cnx: with cnx.security_enabled(False, False): cnx.execute(query, args) The (hopefully) relevant part of the model: class Event(WorkflowableEntityType): organized_by = SubjectRelation( ('CWUser', 'UnlishPage'), cardinality='1*', inlined=True, composite='object') external_url = SubjectRelation( 'Link', __permissions__=__premium_relation_permissions__, cardinality='?*', composite='subject') at_place = SubjectRelation( 'Place', __permissions__=__relation_permissions__, cardinality='?*', inlined=True) class Place(EntityType): refpoint = GeoPoint() The backtrace: ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/session.py:371: in wrapper return func(cnx, *args, **kwargs) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/session.py:381: in check_open return func(cnx, *args, **kwargs) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/session.py:1019: in execute rset = self._execute(self, rql, kwargs, build_descr) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/devtools/__init__.py:814: in new_execute rset = base_execute(*args, **kwargs) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/querier.py:611: in execute results = plan.execute() ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/querier.py:192: in execute result = step.execute() ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/ssplanner.py:374: in execute result = source.syntax_tree_search(cnx, union, args, cachekey, inputmap) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/native.py:552: in syntax_tree_search sql, qargs, cbs = self._rql_sqlgen.generate(union, args, varmap) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:732: in generate sql = self.union_sql(union) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:753: in noparen_union_sql return '\nUNION ALL\n'.join(sqls) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:752: in <genexpr> for i, select in enumerate(union.children)) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:834: in select_sql needalias or needwrap) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:910: in _solutions_sql self._state.add_restriction(select.where.accept(self)) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/rql/utils.py:166: in accept return visit_method(self, *args, **kwargs) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:961: in visit_and part = c.accept(self) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/rql/utils.py:166: in accept return visit_method(self, *args, **kwargs) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:961: in visit_and part = c.accept(self) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/rql/utils.py:166: in accept return visit_method(self, *args, **kwargs) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:961: in visit_and part = c.accept(self) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/rql/utils.py:166: in accept return visit_method(self, *args, **kwargs) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:1057: in visit_relation sql = self._visit_outer_join_relation(relation, rschema) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:1201: in _visit_outer_join_relation outertype, condition) ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:558: in replace_tables_by_outer_join self.mark_as_used_in_outer_join(leftalias, addpending=False) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <cubicweb.server.sources.rql2sql.StateInfo object at 0x7f8620d9c850>, tablealias = '_X' addpending = False def mark_as_used_in_outer_join(self, tablealias, addpending=True): """Mark table of given alias as used in outer join. This must be called after `outer_tables[tablealias]` has been initialized. """ # remove a table from actual_table because it's used in an outer join # chain > scope, tabledef = self.tables[tablealias] E KeyError: '_X' ../../../.virtualenvs/unlish-dev/local/lib/python2.7/site-packages/cubicweb/server/sources/rql2sql.py:522: KeyError | |
priority | important |
---|---|
type | bug |
done in | 3.21.0 |
load left | 0.000 |
closed by | #a1e8dbb7215b [rql2sql] Fix iter_exists_sols() excessive cleaning of _state.tables |
patch | [rql2sql] Fix iter_exists_sols() excessive cleaning of _state.tables [applied] |
Comments
-
2015/06/17 14:27, written by cdevienne
-
2015/06/17 14:30, written by cdevienne
-
2015/06/17 14:30, written by cdevienne
-
2015/06/18 15:43, written by cdevienne
-
2015/06/18 15:52, written by cdevienne
-
2015/06/18 16:16, written by cdevienne
add commentInterestingly, if I remove the "EXISTS()" (not its content, only the EXISTS()", the bug disappears.
Same thing if I remove the '?' on X_EXTERNAL_URL.
In fact, any removal from the query removes the bug (it seems).
Well, the "A is Place" can be removed, the bug remains.
New element: the problem occurs only if the EXIST() clause is first in the list. In second of third position, no problem.
From what I understand, here is what happens:
When visiting the EXISTS, at one moment the X variable gets visited. Because it is the first time, the variable gets a new attribute _q_sqltable and is added to _state.done. As a side effect, _state.tables gets a new member, "_X".
Later, the variable is visited again for external_url, and the join needs the underlying table. But at this point, for some reason, _state.tables no longer has "_X". But because X is still in _state.done it was not re-added to _state.tables.
From the tracing I did, it seems that _state.tables gets emptied when leaving the EXISTS. Which makes sens.
So, I suspect that the EXISTS handling does not clean up enough after doing its job.
Any additional hint/insight is more than welcome.
Continuing my digging.
def iter_exists_sols(self, exists):
This function does, imo something bad. It restores completely the tables event for entries that were added by a variable that is not specific to this exists.