Skip to content

unnest isn't handled correctly #283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jimfulton opened this issue Aug 19, 2021 · 1 comment · Fixed by #290
Closed

unnest isn't handled correctly #283

jimfulton opened this issue Aug 19, 2021 · 1 comment · Fixed by #290
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@jimfulton
Copy link
Contributor

jimfulton commented Aug 19, 2021

Given a table, foo, with schema:
[SchemaField('objects', 'RECORD', 'REPEATED', None, (SchemaField('object_id', 'STRING', 'NULLABLE', None, (), ()),), ())]

>>> foo = Table('foo', metadata, autoload_with=engine)
>>> print(select([func.unnest(foo.c.objects).alias('foo_objects').column]).compile(engine))
SELECT `foo_objects` 
FROM unnest(`foo`.`objects`) AS `foo_objects`

BigQuery needs this select to be spelled:

SELECT foo_objects
FROM `foo`, UNNEST(`foo`.`objects`) as foo_objects

or

SELECT foo_objects
FROM `foo`, UNNEST(`objects`) as foo_objects

The

`foo`

in the FROM is needed to establish an alias for interpreting

`foo`.`objects `

or for providing a default table for

`objects `

But otherwise, there's no special requirement for the extra from.

In particular, consider:

select x from unnest([1,2,3]) as x

which needs no extra from because the expression given to unnest doesn't reference a column.

The fact that unnest takes an expression that may have 0 or more column references (from zero or more tables) makes the fix rather complicated.

(The examples above don't show a dataset, because it's set as a default.)

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. label Aug 19, 2021
@jimfulton jimfulton self-assigned this Aug 19, 2021
@tswast tswast added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Aug 19, 2021
@jimfulton
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
2 participants