SSL `IntFlag` enums `VerifyFlags` and `Options` are not continuous and contain "false" aliases

[jbdyn]

I noticed that the IntFlag enums VerifyFlags and Options from the ssl module are not continuous and contain aliases to (partially) absent members, i.e. "false" aliases.

This gives inconsistencies in flag iteration:

• overshooting

  VerifyFlags

  from ssl import VerifyFlags
  print(~VerifyFlags(0))  # common for "not no flags", i.e. all flags
  # <VerifyFlags.VERIFY_CRL_CHECK_LEAF|VERIFY_X509_STRICT|VERIFY_ALLOW_PROXY_CERTS|VERIFY_X509_TRUSTED_FIRST|VERIFY_X509_PARTIAL_CHAIN|VERIFY_CRL_CHECK_CHAIN|491411: 1048575>
  # with added `491411`

  Options

  from ssl import Options
  print(~Options(0))  # common for "not no flags", i.e. all flags
  # <Options.OP_LEGACY_SERVER_CONNECT|OP_ENABLE_KTLS|OP_IGNORE_UNEXPECTED_EOF|OP_NO_TICKET|OP_NO_COMPRESSION|OP_ENABLE_MIDDLEBOX_COMPAT|OP_CIPHER_SERVER_PREFERENCE|OP_NO_SSLv3|OP_NO_TLSv1|OP_NO_TLSv1_2|OP_NO_TLSv1_1|OP_NO_TLSv1_3|OP_NO_RENEGOTIATION|OP_ALL|28163875: 4294967295>
  # with added `28163875`

• false aliases

  VerifyFlags

  from ssl import VerifyFlags
  print(list(VerifyFlags.VERIFY_CRL_CHECK_CHAIN))  # with value `12`
  # [<VerifyFlags.VERIFY_CRL_CHECK_LEAF: 4>, None]
  # member with value `8` is absent from `VerifyFlags`

  Options

  from ssl import Options
  print(list(Options.OP_ALL))
  # [None, None, None]
  # no foundational members in `Options` at all

• missing members due to being false aliases

  VerifyFlags

  from ssl import VerifyFlags
  
  all_members = set(VerifyFlags.__members__.values())
  iterated_members = set(VerifyFlags)
  
  print(all_members - iterated_members)
  # {<VerifyFlags.VERIFY_DEFAULT: 0>, <VerifyFlags.VERIFY_CRL_CHECK_CHAIN: 12>}
  # value `0` is trivial, but `VERIFY_CRL_CHECK_CHAIN` actually needs to be standalone (see previous bullet point)

  and consequently

  from ssl import VerifyFlags
  from functools import reduce
  from operator import or_
  
  all = reduce(or_, VerifyFlags)  # actually not all flags
  print(all)
  # <VerifyFlags.VERIFY_CRL_CHECK_LEAF|VERIFY_X509_STRICT|VERIFY_ALLOW_PROXY_CERTS|VERIFY_X509_TRUSTED_FIRST|VERIFY_X509_PARTIAL_CHAIN: 557156>
  # missing false alias `VERIFY_CRL_CHECK_CHAIN`
  
  print(sum(set(VerifyFlags.__members__.values())))
  # 557168
  # missing the `VERIFY_CRL_CHECK_CHAIN` with value `12`

  Options

  from ssl import Options
  
  all_members = set(Options.__members__.values())
  iterated_members = set(Options)
  
  print(all_members - iterated_members)
  # {<Options.OP_NO_SSLv2: 0>, <Options.OP_ALL: 2147483728>}
  # value `0` is trivial, but `OP_ALL` actually needs to be standalone (see previous bullet point)

  and resulting implications analogous to VerifyFlags.

With that, enum.show_flag_values also cannot really be used anymore.

I found these related issues:

• Introduction of enums: Use IntEnum and IntFlags in ssl module #72212
• planned removal of deprecated code: [ssl] Remove deprecated protocols, OP_NO_TLS/SSL, and support for TLS < 1.2 #94598

The behavior shown above kind of collides with the achievement for simplicity mentioned in #72212 - and also confused me heavily.

@tiran @ethanfurman What do you think?

[picnixz]

FTR, Tiran is not more active on the repo and @gpshead and me maintain SSL/crypto related-modules.

I don't think we can anything here. The value of the flags are dictated by the libssl implementation so we can't change their values. So I think we should consider those enums as restricted flag enums, and not really real flag enums where you assume that each bit has its own meaning.

[jbdyn]

The value of the flags are dictated by the libssl implementation

Ah, yes, I thought so.

I think we should consider those enums as restricted flag enums

Definitely. The flags itself, however, don't convey that info to the user.

How about introducing a user-facing proxy enum just holding the flag names with unimportant, but continuous flag values?

The actual options with values of the libssl implementation could be set in the SSLContext internally.
The proxy enum could either be additional to the current enums (easy, but with translation API) or replace them by holding both the flag value for bitwise operations and the actual integer value for everything else (harder, as a custom enum class needs to be defined).

[jbdyn]

I imagine something like this:

from enum import IntFlag, auto
from functools import reduce
from operator import or_
from ssl import Options


UserFacingOption = IntFlag(
    "UserFacingOption",
    (
        (name, auto() if member > 0 else member)
        for name, member in Options.__members__.items()
    ),
)


def translate(
    user: UserFacingOption,
    implementation: Options,
) -> Options:
    return reduce(
        or_,
        (implementation[member.name] for member in user)
    )

or via a mixin class:

from enum import IntFlag, auto
from functools import reduce
from operator import or_
from ssl import Options


class Translated:
    target = IntFlag

    def translate(self):
        return reduce(
            or_,
            (self.target[member.name] for member in self)
        )


class TranslatedOption(Translated):
    target = Options


UserFacingOption = IntFlag(
    "UserFacingOption",
    (
        (name, auto() if member > 0 else member)
        for name, member in Options.__members__.items()
    ),
    type=TranslatedOption,
)

[picnixz]

I don't think this is necessary for the stdlib. It can live in user-code or it can be a generic helper in enum. At least, IMO, it's not worth doing this for SSL.

Having two different enumerations for flags is, IMO, not a good idea as it may confuse readers. Usually, you're not meant to iterate over those flags, you're meant to use them as is and explicitly.

[jbdyn]

Fair.

Usually, you're not meant to iterate over those flags, you're meant to use them as is and explicitly.

For static cases, sure, but for dynamic access like extracting the names of set flags for display or storage (also a form of being explicit), iteration is needed. I don't know whether one is more common than the other.

That being said, I just don't like to inconsistent behavior in this security context, as it feels too lax for me here.
When Options and VerifyFlags offer the functionality of (their indirect superclass) Flag, I expect it to work flawlessly.
If the user is really meant to use those flags explicitely, an enum class other than Flag would be a better fit to enforce that. Or a note in the docs about irregular flag values and their implications.

[jbdyn]

@ethanfurman You seem to maintain the enum module. Is that (still) correct?

If true: What do you think about extending flags to also support discontinuous flag values?
One would need to rework the "not zero" case and the definition of aliases, but would also be able to deal with this kind of situation.

[picnixz]

I prefer that we close this specific issue and re-open another one targetting your request. You can however link it as past discussion. However, please check if we don't already have this kind of issue in the past (and that may have already been rejected)

[jbdyn]

Okay, let's do that. Thank you @picnixz!

[ethanfurman]

@jbdyn , yes, I still maintain the enum module. I suspect that enums can already handle the desired behavior -- the question is: what is the desired behavior in a security context? Flags can automatically discard unnamed bits, or raise an exception if unnamed bits are present -- which, if either, of those behaviors would be appropriate? (This question and it's answer should be in the new issue.)