Validate column types against the whole string, not a prefix

[roed-math]

First of five PRs from the August 3 security audit. It is the one that fixes a
concrete injection, so it is worth landing before the others.

The problem

A column type has to be interpolated into CREATE TABLE / ALTER TABLE as SQL
text: PostgreSQL has no placeholder for a type. The check guarding that
interpolation used regexp.match() against patterns that were not anchored at
the end, so any string beginning with a valid type validated with the rest
still attached, and _order_columns() then interpolated the caller's original
string. psycopg sends a parameterless statement with the simple query protocol,
which executes every statement in the string.

I confirmed this end to end against a disposable database on the unpatched tree:
db.create_table(name, [("c", <type with a second statement appended>)])
returned normally and the appended statement had run.

The entry points that take a type from outside psycodict are create_table,
add_column, and the column types read from a data file header by
reload(adjust_schema=True) and reload_all(adjust_schema=True). The last two
matter most in practice, since those types come from a file rather than from a
call site the administrator just typed. (create_table_like takes its types
from the source table's catalog entry, so it was not a way in; it goes through
the same validator now regardless.)

The character-type pattern was separately malformed — a bracket typo in the
varchar length (\(1-9][0-9]*\)) meant varchar(16) never matched it and was
only ever accepted as a prefix of varchar.

The fix

• One authoritative validator, psycodict.base.validate_column_type(typ),
  returning (sql_spelling, storage_cost). It requires a string, strips
  surrounding whitespace, rejects anything outside an ASCII character set that
  excludes NUL, control characters and non-ASCII lookalikes, and matches with
  fullmatch().
• Every type-bearing DDL path emits the returned spelling rather than the
  argument: _order_columns, _create_table, _create_table_from_header,
  add_column, and the temp table built when resorting ids. _get_type_sortkey
  and _check_col_datatype are now thin wrappers over the same validator
  instead of a parallel matching loop.
• The char family is described deliberately (char, character, varchar,
  character varying, each with an optional length, and an optional collation)
  instead of by a typo'd pattern. Keywords match case-insensitively; collation
  names do not, since they are quoted identifiers — "C" is a collation and
  "c" is not, so the spelling that goes into the DDL is the spelling that was
  validated.
• Fixed types are emitted from a closed mapping of preconstructed SQL objects,
  so the common case interpolates a constant rather than a caller-supplied
  string.
• add_column validates before it mutates col_type, and updates the in-memory
  schema only after the DDL succeeds.
• Invalid types raise InvalidColumnTypeError. It subclasses ValueError (the
  natural type for a bad argument) and also RuntimeError, which is what an
  invalid type raised before, so existing except RuntimeError callers keep
  working. Two existing tests assert RuntimeError and are unchanged.

Tests

New tests/test_security.py (106 cases):

• every type the grammar accepts is also created for real on the server, so the
  grammar cannot drift away from PostgreSQL's and start rejecting a column an
  existing database already has;
• the adversarial list from the audit — appended statements, comments,
  varchar(0), varchar(-1), a NUL byte, a Cyrillic lookalike — is rejected;
• create_table (both a column type and the id_type), add_column,
  reload(adjust_schema=True) and reload_all(adjust_schema=True) are each
  driven with a type carrying a statement that would create a uniquely named
  marker table. Each test asserts the exception and that no marker exists,
  that the target relation or column was not partially created, and that the
  table object's col_type / search_cols were not mutated.

Full suite: 972 passed, 36 skipped against PostgreSQL 18.

Because the repository is at 1.0.0rc1, a patched release candidate before 1.0.0
final would be the natural home for this; I left the version alone.

🤖 Generated with Claude Code

[read-the-docs-community]

Documentation build overview

📚 psycodict | 🛠️ Build #33884175 | 📁 Comparing 66ca07b against latest (85283b7)

  🔍 Preview build  

6 files changed · ± 6 modified

± Modified

• genindex.html
• api/base.html
• api/database.html
• _modules/psycodict/base.html
• _modules/psycodict/database.html
• _modules/psycodict/table.html