Forward references in methods

[superbobry]

Hi everyone,

How should a type checker resolve the return type of f in

import abc

class A:
    def abc(self) -> 'abc.ABC': ...

?

According to PEP 484

The local and global namespace in which it [a forward reference] is evaluated should be the same namespaces in which default arguments to the same function would be evaluated.

which, I think, implies that the reference abc in the return type of f should be resolved to abc the method (in the class namespace) and not abc the module in the global namespace. Does that sound right?

---

A few more motivating examples

class A:
    C = int
    def f(self) -> C: ...

class A:
    def f(self) -> C: ...
    C = int

[erictraut]

That's correct. Mypy and pyright both interpret it this way. However, I'd recommend against using conflicting names like this. It harms readability, and you will inevitably see inconsistencies across tools.

[superbobry]

PEP 484 suggests that abc should be evaluated (or resolved) in the same namespace where the method is defined. I imagine a type checker could implement that by finding all assignments to abc reaching the exit from the class body. Clearly, this is not what existing type checkers do. Could you comment on how pyright does the evaluation in this case?

[erictraut]

Pyright used to do as you suggested, but it was pointed out that this doesn't match the runtime behavior when evaluating forward references. This created problems when working with runtime type checkers. I think all of the type checkers and the runtime are now in alignment. The runtime behavior is clearly documented in PEP 563.

As I mentioned above, I encourage developers to avoid this type of namespace overlap because it makes the code fragile (e.g. when removing or adding a from __future__ import annotations to the file, the behavior changes in an unexpected way), and you may see different behaviors from different tools.

[superbobry]

Thanks Eric, this is very helpful.

One remaining issue is the discrepancy between PEP 563 and typing.get_type_hints. PEP 563 says that

For classes, localns can be composed by chaining vars of the given class and its base classes (in the method resolution order).

and then gives multiple examples of valid and invalid forward references, including

class A:
    T = int
    def method(self) -> 'T': ...

(I have adapted the second example from the "Backwards compatibility" section)

which, however, does not work with typing.get_type_hints

>>> typing.get_type_hints(A.f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/typing.py", line 1496, in get_type_hints
    value = _eval_type(value, globalns, localns)
  File "/usr/lib/python3.9/typing.py", line 292, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "/usr/lib/python3.9/typing.py", line 554, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'T' is not defined

Could you advise how to think about that? What is the general rule for resolving a forward reference?

[hmc-cs-mdrissi]

The rule is you need to pass locals/globals yourself. typing.get_type_hints full signature is (obj, globalns=None, localns=None, include_extras=False).

For code that inspects type annotations inside of class/function scope it needs to keep track of right namespaces to pass in. I have some code that uses decorator for inspecting type annotations of a class and when that decorator is called it looks at caller's stack frame for it's namespaces and saves them for type evaluation later. Alternatively you could have a function that registers forward references in some dict that you reuse as namespace later. I use that to handle case for type aliases I need to evaluate in a different module later.

A lot of runtime type inspection libraries have edge cases for situations like this as managing it correctly is a bit messy and is easiest done by forcing explicit registration elsewhere.

[superbobry]

Thanks @hmc-cs-mdrissi and apologies for the confusion! My question was about the rule a type checker should use to implement evaluation of forward references.