Skip to content

[sqlalchemy-spanner]: reflected primary keys have no "name" key, so Alembic >= 1.18 autogenerate fails with KeyError: 'name' #18841

Open

Description

@rocksnow1942

Determine this is the right repository

  • I determined this is the correct repository in which to report this bug.

Summary of the issue

SpannerDialect.get_pk_constraint() and get_multi_pk_constraint() return primary keys as {"constrained_columns": [...]}, without the name key that SQLAlchemy's ReflectedPrimaryKeyConstraint requires (name: Optional[str], from ReflectedConstraint). Since Alembic 1.18.0, autogenerate reads that key, so alembic revision --autogenerate and alembic check fail with KeyError: 'name' on any Spanner database that has a table.

API client name and version

sqlalchemy-spanner 1.20.1 (also main at eb8a766)

Problem description

Alembic 1.18.0 (sqlalchemy/alembic#1771) changed autogenerate on SQLAlchemy 2.x to load the reflection data of all tables in a schema with the get_multi_*() inspector methods. It then wraps each reflected constraint name in conv(), which reads const["name"] from the primary key, foreign key, index and unique constraint dicts (_apply_reflectinfo_conv in alembic/autogenerate/compare/util.py, unchanged through 1.20.1). The primary key dicts from get_multi_pk_constraint() have no name, so this raises KeyError. Every Spanner table has a primary key, so autogenerate fails as soon as the database has a table.

Alembic 1.17 and earlier reflect one table at a time through Inspector.reflect_table(), which reads the name with .get("name"), so they don't hit this.

The compliance suite doesn't catch it, because the Spanner override of _check_list in tests/test_suite_20.py only compares keys that are in both the result and the expected dict.

Reproduction steps: code

Run against the emulator with alembic>=1.18 and sqlalchemy>=2.0:

import os

os.environ.setdefault("SPANNER_EMULATOR_HOST", "localhost:9010")

import sqlalchemy as sa
from alembic.autogenerate import compare_metadata
from alembic.ddl.impl import DefaultImpl
from alembic.migration import MigrationContext
from google.cloud import spanner


class SpannerImpl(DefaultImpl):
    __dialect__ = "spanner+spanner"


client = spanner.Client(project="emulator-test-project")
instance = client.instance(
    "test-instance", f"{client.project_name}/instanceConfigs/emulator-config"
)
if not instance.exists():
    instance.create().result()
database = instance.database(
    "pk-name-repro",
    ddl_statements=["CREATE TABLE singers (id INT64 NOT NULL) PRIMARY KEY (id)"],
)
if database.exists():
    database.drop()
database.create().result()

engine = sa.create_engine(
    "spanner+spanner:///projects/emulator-test-project/instances/test-instance"
    "/databases/pk-name-repro"
)
metadata = sa.MetaData()
sa.Table("singers", metadata, sa.Column("id", sa.BigInteger, primary_key=True))

with engine.connect() as connection:
    print(sa.inspect(connection).get_pk_constraint("singers"))
    print(compare_metadata(MigrationContext.configure(connection), metadata))

Reproduction steps: actual results

{'constrained_columns': ['id']}
Traceback (most recent call last):
  ...
  File ".../alembic/autogenerate/compare/tables.py", line 193, in _compare_tables
    _InspectorConv(inspector).reflect_table(t)
  File ".../alembic/autogenerate/compare/util.py", line 216, in _make_reflection_info
    table_key: self._return_from_cache(
  File ".../alembic/autogenerate/compare/util.py", line 113, in _return_from_cache
    return self._apply_reflectinfo_conv(individual)
  File ".../alembic/autogenerate/compare/util.py", line 181, in _apply_reflectinfo_conv
    if const["name"] is not None and not isinstance(
       ~~~~~^^^^^^^^
KeyError: 'name'

Reproduction steps: expected results

{'name': 'PK_singers', 'constrained_columns': ['id']}
[]

OS & version + platform

Linux

Python environment

Python 3.13

Python dependencies

sqlalchemy 2.1.5, alembic 1.20.0, google-cloud-spanner 3.71.0, sqlalchemy-spanner 1.20.1, Spanner emulator 1.5.58

Additional context

Spanner names primary key constraints PK_<table>, so get_multi_pk_constraint() can select tc.constraint_name along with the columns and return it as name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: spannerIssues related to the Spanner API.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions