Then you will not get this error (NameError: name 'Sequence' is not defined) because it will not need the reference to `Sequence` for the annotation but the annotation is simply a string.
Using `if TYPE_CHECKING` is also useful when you want to speed up the module load time by not importing unnecessary modules (unnecessary at module load time).
And then, also to resolve import cycles, where you still want to annotate the function argument types.
One problem though: If this is a type/object which is not needed for module load time (there it's just used to annotate some function arg type), but which will be needed inside the function, type checkers and IDEs (e.g. PyCharm) will not show any problem, but then at runtime, you will the NameError once it tries to use the type/object.
Example module:
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import torch
def agi_model(query: torch.Tensor) -> torch.Tensor:
torch.setup_secret_agi()
...
This module will load just fine, because `torch` is only needed for type checking at module load time. However, once you call this `agi_model` function, it will fail because `torch` is actually unknown at runtime. However, no IDE will show any problem with this code. (Edit: Eclipse/PyDev handles it correctly now, see answer below.)
Then you would add another `import torch` inside that `agi_model` function. However, then the IDE might even complain that this hides the outer scope `torch`.
> However, no IDE will show any problem with this code.
That's not entirely True... I just checked it here and Eclipse/PyDev does flag this correctly -- since version 11.0.0 ;)
See: https://www.pydev.org/history_pydev.html (Imports found inside a typing.TYPE_CHECKING will be considered undefined if the scope that uses it requires it to be available when not type-checking).
It kind of works, you just have to use `typing.get_annotations(func)` instead of `func.__annotations__`. Although if you aren't importing the types used in your annotations that isn't going to work... I would guess that's rare in practice.
If you define a class inside another class or a function, get_annotations() will never be able to resolve it and just return a string, which is not very useful.
Not really. `import type` just means that this import is guaranteed to be removed when compiling to JS. TS already fully erases types during compilation, and this is just a way to guarantee that the imports of types are guaranteed to be erased as well.
E.g. for `import { SomeType } from "backend-server"`, you don't want to include your entire server code in the frontend just because you imported the response type of some JSON API. `import type` neatly solves this issue, and it even enforces that you can't import anything but types from your backend.
I think in practice that increases the need for TYPE_CHECKING. If you use deferred annotations, and you don't import the types you use, type checkers and IDEs will complain, and go to definition won't work. But if you import the types outside of a TYPE_CHECKING block it's wasteful, since you don't actually need the import for your code to run. So you need TYPE_CHECKING to satisfy your linter without impacting runtime performance.
> However, Python doesn’t have the compile-time check, because it’s an interpreted language that is dynamically-typed, which means its only real place to check is at runtime
Pet peeve: Python is compiled (into bytecode), so it could theoretically do checks at compile time. The "dynamically-typed" part is correct and is the real reason.
The problem for Python with type checking is that the language has essentially no compile-time constructs at all, just a bunch of assignment operators with different syntax: `def` assigns a function object to a variable when it's executed, `import` assigns a module, `class` assigns a class object. This means that everything involving names in Python is just a variable lookup (incidentally, this is why a function call must follow the declaration in Python: the variable is unbound if you haven't executed the function definition yet). The reason `Sequence` fails in the example in the article is simply that the code is trying to read an unassigned variable, no different from writing `vairable` instead of `variable` in your code.
TS is not interpreted. It compiles to JS, which is interpreted, but TS itself is not. There are no TS interpreters, they all compile to JS behind the scenes.
Or does Deno run TS natively? At a glance it looks like even its runtime is written in JS[0].
EDIT: looks like they have bindings for V8[1] so yeah, just a JS interpreter behind the scenes.
Likewise, the Python typing annotation language could very well be on the compiled side...
Someone took the interpreted JavaScript language, added typing syntax on top, and typing can be checked statically.
The Python team took the interpreted Python language, added typing syntax on top, and here we are reading a claim that it can't be checked statically because it's interpreted. You understand my puzzlement.
It is not so much that it is fuzzy, it is that the two definitions are not exclusive.
A compilation translates a language to another, whether it is native code, internal bytecode, portable bytecode, or another source language. Compilation might also type check and optimize depending on the language. Interpretation executes a language; normally bytecode is executed, but pure source level interpreters exist. Native code is "interpreted" by the cpu itself of course.
For example JAVA is certainly a compiled language, but it is typically compiled to bytecode and then interpreted or JITed (which is a combined interpreter/compiler).
Small but important correction required in point 3 of the TL;DR:
> Python doesn’t care about types at runtime
This should say "Python doesn't care about type annotations at runtime."
Python _does_ care about types at runtime - but it doesn't use type annotations to compute them.
That doesn't detract from what's otherwise a really clear and helpful post. Python is living through a schizophrenic period in its evolution, and it's causing some problems. I hope it gets ironed out, though don't underestimate the difficulty. Python remains my go-to for many problems: a joy to use (for me) in many cases. But I definitely feel the need for static typing as codebase size increases. Having type annotations is good, but not being able to rely on them (i.e. static typing is not sound) adds to cognitive load and detracts from a confident development experience.
I will add that, in my experience, having type annotations that are unreliable, cumbersome, with so many edge cases and requiring this kind of magic (e.g. "if TYPE_CHECKING", etc) which is beyond most "casual" Python users effectively means type annotations are a very hard sell for most non-hardcore team mates.
I've been in a position to try improving processes and code quality of the Python codebase, and introducing mypy and type annotations has always been a very hard sell. The team just won't accept the benefits, given the difficulties in using it. If only Python made it easier and more reliable, but the current state is a mess.
I come from the static typing world, so of course I see the benefits. But most/all of my team mates don't, and so they see this as some really odd hoops I want them to jump through, for minimal benefit (in their eyes).
I think type annotations are extremely valuable even if you don't use mypy. On my team we basically use them as comments that are understood by the IDE, but aren't enforced. Of course comments can lie, but appeasing mypy really is a lot of work, and I would generally rather have a slightly dishonest but mostly correct type hint than some TypeVar monstrosity or, worse, Any.
Hm. Type annotations as comments is the ultimate annoyance to my coworkers. They really do see them as "what's the point" in that case. Plus, like you say, they are misleading. The look like code, but they don't do anything.
I think if you're already documenting function argument types, there is no point. But in my experience it's unworkable to ask people to write full documentation for every function, but requiring type hints is not as hard and is nearly as useful. It is a little weird that they look like code, but I think you get used to it.
It's arguable whether a SyntaxError happens as a compile-time check. It's true that it is raised before runtime. But a document that can be parsed according to the syntax definition is a necessary precondition to all checks of any static program property. Syntactic correctness in itself is not a static property of a program. A program is, by definition, syntactically correct.
Without a correct syntax there is no way to assign meaning or execute or interpret, etc.
One question comes to mind: isn't cyclic dependencies something to be resolved at the root cause (remove the cycle)? It's an issue that causes various pains down the line, like type checking in this case.
The issue is that, often, without type annotations there would be no cyclic dependency (because any type matching the implicit interface would work). So by introducing type annotations (which means extra imports), you end up having to refactor quite a lot of code and the end game is a one-type-per-module-with-separate-interface-definition system of the sort encountered in, well, statically-typed languages. If you don't do this you end up with the 'Type in string' annotations mentioned in the article and TYPE_CHECKING-gated imports. Both of which feel kind of hackish.
It's fine to have implicit circular dependencies in Python because duck typing means that every function is effectively generic. Type annotations eliminate this and turn Python into a fundamentally different language.
Type annotations aren't just extra boilerplate in function signatures, they also have these sorts of knock-on effects.
Broadly speaking, yes, but for some situations like database models with bi-directional relationships that can cause unnecessary maintenance burden simply for the benefit of type checking which this allows you to work around since the cyclical nature is only caused by typing not runtime semantics
Removing cycles may require extensive refactoring of the code, sometimes resulting in unintuitive layouts. All, just to solve the cyclic-import errors.
At the core of the problem is the fact that there's no thing as "partial import" in Python. When you do "from M import A" in Python, all the contents of M are evaluated and a reference to A is added to the current namespace. So, cyclic dependencies arise sooner or later, unless you adopt some very un-pythonic style for your codebase (e.g. one file per class).
In python it's very hard to avoid cyclic dependencies. Something as simple as a parent-child link between two classes is a cyclic dependency, and if the classes are in different files you have a car like in the blog post. This comes up especially hard when typing your codebase, because you have to name every type, and in python implementation is the interface, i.e. you can't just include a type definition file.
I also wonder. If a child class knows its concrete parent (apart from where inheritance is declared), and not just references it via super(), things have already gone badly wrong.
Could you expand on it? It is truly surprising for me that anyone would find code with type annotations to be significantly worse, for any reasons whatsoever.
On the contrary, I joyfully read and write code with type annotations. It is obviously very useful knowing which object types a function expects and which it will return.
Professionals should stop putting up with unprofessional tools and languages. This? This whole mess is not professional. If you want types, do not use python. Python is not a typed language. You can't just bolt on a type system, as is being live demonstrated here.
It's reasonable and "professional" to want dynamically typed scripting language, for things where real types are a burden not a benefit, which also has libraries that work properly.
For me, that's the benefit of typing in python (as shonky as it is) -- you can use it to help write better library code, then ignore the types (except as accurate documentation) when using those libraries.
Maybe there are other languages that can do this, but the ones i know all have their own downsides.
There is, in my opinion, very little value in a type system that allows types to just be ignored. Or even a negative value, because it instills a false sense of security, because it does not actually prevent any of the mistakes that a type system is supposed to prevent.
If it catches bugs before run-time, that's a benefit to me, in the same way that testing doesn't catch all bugs (and yes, can introduce a false sense of security), but most people still think it's a good idea.
You obviously don't get the haskell experience, where if your program type checks it's probably correct, but the alternative (if, remember, you want a dynamically typed scripting language) is no type checking, which is definitely worse for some people.
I think a type system might be successfully bolted on on Python. I think gradual typing might be made to work. I have no opinion on whether the current type system is good or not.
I do strongly believe that leaving the type checking to a third party program is a terrible idea. I also believe that it should at least be possible to opt-in to type-checking the annotations at runtime during execution (at the very least for tests).
We're not professionals. We don't profess anything. Whether we want software development to become a profession is a huge topic on its own. Professions come with a ton of strings attached and oversight. Something I would personally welcome, but know most of us wouldn't.
But it is its own thing that compiles down to javascript, and is otherwise syntactically and semantically "very very similar" to javascript. It IS always statically typechecked and has good, or at least well defined, semantics for typing. Typescript does not change base javascript. It really is a well separated layer on top of javascript.
The way python does it is different. There's no "typethon". The python grammar was simply extended to allow any random expression as a "type".
In fact:
x: print("hi") = 4
is valid python, and it does print "hi" without even so much as a warning. Personally, I don't think "print(hi)" should be a valid type.
This feels like splitting hairs -- python and typescript are equivalent, as they are both dynamically typed languages with type annotations.
The fact that the typescript implementation does not run typescript directly, and cPython does not type check before running the code, are both implementation details.
Typescript is just a better type system on top of javascript than python's type system is on top of untyped python, because typescript is a cohesive design rather than a collection or random stuff accumulated over time.
There's no reason that couldn't have happened for python, it just didn't.
I don't see what's so difficult. If it was technically possible to run programs that have type errors in other languages, they would have the option to ignore type errors too, because it's convenient.
Typescript also ignores errors (generating js output for you to run) by default, so what's the point if it's not enforced?
Typescript "compilation" is just stripping out types, mostly. JavaScript is a subset of TypeScript (mostly). There was even a proposal to extend JavaScript comments syntax, so all typescript types would become comments (so you could just run typescript code with JS interpreter).
> Personally, I don't think "print(hi)" should be a valid type.
And it isn't:
a.py:1: error: Invalid type comment or annotation [valid-type]
a.py:1: note: Suggestion: use print[...] instead of print(...)
Found 1 error in 1 file (checked 1 source file)
I've had to work on a PHP project recently and it's horrible. Mypy has many pitfalls and it's not great, but at least it's entirely optional, so you can work around the issues or ignore some types until you can fix it or do some refactoring. Typescript might be a different language but it's the best implementation of adding types to a language I've seen so far.
> I've had to work on a PHP project recently and it's horrible. Mypy has many pitfalls and it's not great, but at least it's entirely optional...
Do you mean mypy is optional?
At the risk of opening a can of worms here, what's "horrible"?
In your own PHP code, you can get by without any typing at all. If/when you start to use 3rd party libraries, that may become a factor, though off the top of my head I can't think of a show stopper.
I'm pretty sure that php has also explicitly broken backwards compatibility to achieve this. In general php has become far more strict over the years, which I think is a good thing. And they definitely at least sat down and thought about how to do types before adding grammar rules that allowed any random expression as the type, which is now the case in python.
The backwards compatibility has broken some - I think mostly visibly noticeable between 5.6 and 7.0. And every release there's some deprecations introduced that will eventually break in future. Many complex applications in 5.6 won't run 100% error free in 8.2, for sure. I'm working with a company to move from 5.6 to 7.4 and there's very little that doesn't work in the main code. Their earlier version of Doctrine was using a class called 'Null', and internally (unrelated to doctrine) they had a class named 'Null' - those had to be renamed. That took all of about 2 minutes. Upgrading Doctrine version got us to a version that removed the Null class name there too.
Python also broke backwards compatibility with their v3 release years ago. Unsure why breaking backward compatibility is being called out here(?).
But yeah, overall, it's struck me that PHP has introduced more and stricter typing rules with surprisingly little impact (compared to what could have been, of course).
Using `if TYPE_CHECKING` is also useful when you want to speed up the module load time by not importing unnecessary modules (unnecessary at module load time).
And then, also to resolve import cycles, where you still want to annotate the function argument types.
One problem though: If this is a type/object which is not needed for module load time (there it's just used to annotate some function arg type), but which will be needed inside the function, type checkers and IDEs (e.g. PyCharm) will not show any problem, but then at runtime, you will the NameError once it tries to use the type/object.
Example module:
This module will load just fine, because `torch` is only needed for type checking at module load time. However, once you call this `agi_model` function, it will fail because `torch` is actually unknown at runtime. However, no IDE will show any problem with this code. (Edit: Eclipse/PyDev handles it correctly now, see answer below.)Then you would add another `import torch` inside that `agi_model` function. However, then the IDE might even complain that this hides the outer scope `torch`.