Skip to content

Unnest Function In BigQuery SQLAlchemy Plugin Joins Twice When Selecting Other Columns in CTE or Subquery  #368

Closed
@mgtcardenas

Description

@mgtcardenas

From a customer support request:

Issue summary: This issue is closely related to Unnest function in BigQuery SQLAlchemy plugin joins twice when selecting other columns #353. After the previous issue was resolved, queries using the UNNEST function that are not in Common Table Expressions (CTE) or subqueries work as expected. See the following minimal example:

from sqlalchemy import create_engine, MetaData, select, func as F, Table, Column, Integer, ARRAY

metadata = MetaData()
table = Table(
  "table1",
  metadata,
  Column("foo", Integer),
  Column("bars", ARRAY(Integer))
)

engine = create_engine("bigquery://", future=True)
q = select(table.c.foo, F.unnest(table.c.bars).column_valued("bar"))
print(q.compile(engine))

This outputs the following, which is expected:

SELECT `table1`.`foo`, `bar` 
FROM `table1`, unnest(`table1`.`bars`) AS `bar`

However, if we use the table as a subquery or CTE, the issue manifests again. The following code (continuing from above):

q = q.cte("cte")
q = select(*q.columns)
print(q.compile(engine))

Produces the next output:

WITH `cte` AS 
(SELECT `table1`.`foo` AS `foo`, `bar` 
FROM `table1`, `table1` `table1_1`, unnest(`table1_1`.`bars`) AS `bar`)
 SELECT `cte`.`foo`, `cte`.`bar` 
FROM `cte`

Note that table1 is duplicated in the FROM clause again.

Expected behavior: It is expected to generate the following query instead (note the different FROM clause):

WITH `cte` AS 
(SELECT `table1`.`foo`, `bar` 
FROM `table1`, unnest(`table1`.`bars`) AS `bar`)
 SELECT `cte`.`foo`, `cte`.`bar` 
FROM `cte`

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: bigqueryIssues related to the googleapis/python-bigquery-sqlalchemy API.priority: p3Desirable enhancement or fix. May not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions