Implement in place YAML encrypting/decrypting

[sylr]

I think there is a great lack of a tooling which would allow in place encrypting/decrypting of YAML data.

Working the Ops side of DevOps I have a lot of YAML (mostly kubernetes manifests) with sensitive data I'd like to encrypt so that I can give access to the repos holding those manifests to my whole R&D.

Implemented in this PR:

• Choose value rendering with tag attributes
• Comment support for encrypted values
• Anchor support
• Multiple documents support

$ cat test.yml
---
hey1: !crypto/age This is a string
hey2: &hey2 !crypto/age:SingleQuoted This is a single quoted string
# This is a head comment
hey3:
  subhey: !crypto/age:DoubleQuoted "This is a double quoted string" # this is a line comment
  # This is a foot comment
hey4: !crypto/age:Literal This is a literal string
hey5: !crypto/age:Folded This is a folded string
hey6: *hey2
---
hey1: !crypto/age:NoTag This is a string with no tag
hey2: !crypto/age:SingleQuoted,NoTag This is a single quoted string with no tag
hey3: !crypto/age:DoubleQuoted,NoTag "This is a double quoted string with no tag"
hey4: !crypto/age:Literal,NoTag This is a literal string with no tag
hey5: !crypto/age:Folded,NoTag This is a folded string with no tag

$ age -R ~/.ssh/id_ed25519.pub -y test.yaml
hey1: !crypto/age |
  -----BEGIN AGE ENCRYPTED FILE-----
  YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSB1dkZV
  bWVJblU3VGpGdGY3Rlk0R05DRUdrQkJUUHVDMDVmYTlJQ21rckVrClk5SWRRamZJ
  cGRMc0pNdk1oa0lycTBxYVRtNkgxYnQ1cXJFdjN2TC9FRzAKLS0tIFkyQ3FrNmpX
  R1pSRjhGMkwraDdxT3pzMFJJMjFpTTBIdVZITzNySnpIUkUKi9uihAkgoz5Y4X2y
  6rfcnN4pOEJU2s5fLCqBAo7ByNeqzMja6jNVuh9bPV885yMn
  -----END AGE ENCRYPTED FILE-----
hey2: &hey2 !crypto/age:SingleQuoted |
  -----BEGIN AGE ENCRYPTED FILE-----
  YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSB4N2Y3
  bDhmaG8xVkZXTUU2NjRISEc2VVBHbGRiRFNudGVKWjZGYTZEMlFFCjRQRlY2NTV1
  dUpkV0Z2UU1lSThNKzlFN1ZFUUdadStNVHNlaVBZVGZqZTgKLS0tIDdTRmY5REsv
  WXRaNVJ1UmczOE4rU2VkZHAzOXlXUWpEU3plRE5qMnRUWm8KiVzlGERdxQZXoMi9
  g0ZAF3nyHC6IzFbN5zt4oXoqxS5+QQjvGY4Jly14MLoBB5/8UhoUKbT1dMLmcNyZ
  6W0=
  -----END AGE ENCRYPTED FILE-----
# This is a head comment
hey3:
  subhey: !crypto/age:DoubleQuoted "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSBQU3I0\nVUZRekRpL05EeEZKckJuTzVMT1ZZZlpKS1BCWkFmSWZyallpZ0I0ClE5R21QbWFL\nVUxzUzVqblpSckhzam1rWnVRYkFoZThiWXlMd1l0RVhVZWcKLS0tIGthdmpjdEZV\nLzNSdTFTRjdlalZwN1RFZzREV0FSOGhDUGt1bFRPQ2FsaW8KRofY20wdmWl1Qpsl\npJlNAz0RO0dAuk0TYJVwL6pmb72w0e3kUCApw0l0u/LZC3ZpTfhEmWuQO/sSWoOL\nD5g=\n-----END AGE ENCRYPTED FILE-----\n" # this is a line comment
  # This is a foot comment
hey4: !crypto/age:Literal |
  -----BEGIN AGE ENCRYPTED FILE-----
  YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSB4MjAr
  YzFxQ1VpTlVEek9EMnZzL2hXSWpGbHExMHpGbFU0OHY5cVpoczFjCjFGS29kdXJB
  UGR6eTBGYm0wMVpzd3VkcjVOb0ptSzNSZmNkZUpxVW14YWMKLS0tIEpOSjUxVWJR
  NVp0NFBMallscXNnZlI0bGp0THlXTHpKUTRVUUt3N3ZkVkkK7snrM/VLPqIzr4sd
  CVcKteGV75hPVCfd05lDtMzlX88hBfCCSQKKnY0E7NNpaLoIirFKDrBa7F0=
  -----END AGE ENCRYPTED FILE-----
hey5: !crypto/age:Folded |
  -----BEGIN AGE ENCRYPTED FILE-----
  YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSBKNUlO
  RkhyZVArQ1QzT1F1Vk5ZcWV4QkpsRC82L2V1WkQ3bDBuMWFJblM4CkFCWVEwMkY0
  V1NNZi9ta0c3NVJJYkJvMnExakxVVS9ra0w2QklLMEdIbFkKLS0tIFk0RkJjdS9l
  ODFOanpIZ0RMMzNiVm9jcEFOTUlMblE4QVc5UWdacmtGUTAKzDRsZUr/bdAoOqQ+
  MC36ykLkRcJEJ/06+McBAe9T1lpqursExTFj7ePVHO15vBkBm1O0d8UDRw==
  -----END AGE ENCRYPTED FILE-----
hey6: *hey2
---
hey1: !crypto/age:NoTag |
  -----BEGIN AGE ENCRYPTED FILE-----
  YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSBwTGJW
  LzFJS0xWdklVQkJrUHZva2hCU2NCRXVBeEhCM1hsTnhDTVFubEFRClc0bTJnandX
  bTJnQVh0NTFpWEU3eVRzQTVLdEFnc09XSWpmSGk1dW12d00KLS0tIDNHSExLWHBH
  RCsybWZPY0czN2Z2UkJCd052VU91RVVrVm9KaUJYaTVIRDAK/cHulkevVFgQHe+h
  kAH9JPWtE3v+X024I0sHHhFuSo4XDCfBJTevwurJasYrL9Et680pEO1xKHReGD0G
  -----END AGE ENCRYPTED FILE-----
hey2: !crypto/age:SingleQuoted,NoTag |
  -----BEGIN AGE ENCRYPTED FILE-----
  YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSB1NkR4
  L2xkTG1xQm9WeWQ3MXJYMGNUWGNvODVTZ0hJa3ovcHhRdk9iMjN3CkxRSVRXTGk3
  d2FVR0VmU1pPYXV0UUhSd2w0NUtFSy9wQ0RaUTJkMzAwWDgKLS0tIHZycVljS1VI
  L0t3M3RoV1NrWU52ODlhUEpmQlc4ZCsycHcrM0NWSVR3bVEKDJRL89scCx2v88B8
  OXQAP4hpFc8kaR6DAeYkxkco+huF2ZQyH+9h32YReT6LDeBpHbkxXq2nlkXr5VCT
  vRq/rnvJJlDHRyAFkfY=
  -----END AGE ENCRYPTED FILE-----
hey3: !crypto/age:DoubleQuoted,NoTag "-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSBSVVFY\nZ3U0bGxuQ2lscFNkRUp5YThvV2xJWGtlUjEwQWd4TWtjZ29reVdNCk4vc2hqTDJp\nSXpkQy9iTkdyYVczeHVseU11RmNsZlNXV1BsQ1hTOVJhUGsKLS0tIHBKVjFPOXpu\nU1pNTGZpNEhlWHdsbTZWUFJaVjBVZGprN0ZJTmtHQ2VPOWsKtpS3yiSQaTDXkCVj\nqaA6wQCRYCYc05ehZpz8ytavnLoKKc5NTMm/N2qeQ2AKxAJuX0T29lcZzl+2b9F9\n2Uu4L7tf8fMMm3+SKpk=\n-----END AGE ENCRYPTED FILE-----\n"
hey4: !crypto/age:Literal,NoTag |
  -----BEGIN AGE ENCRYPTED FILE-----
  YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSBmRDJy
  VXNWS0duNHY1RHgxZmJvWHF4OWhoMnkxeWlNcTBiRzJpdGpkb1dvCnNNYkpzSnJG
  K3k3aEwwUitTYW4yUTBCL2p6L0xBeEx0NHYwc2dkQ3dGcTQKLS0tIEZmcDhmOHo5
  NkZiM2gyWFBFdk4zdytEQkcyRU5Bam9qVkNCdGdLYmRFUkEKW2eJX+SRo54Dzm0y
  3a4FyaanMHqzButmkMLm4eQyPZOzTX/Nzc6Zi5GPCtATGKFdDjckDNMwfp2CKF+P
  fuK7aKqW6Eg=
  -----END AGE ENCRYPTED FILE-----
hey5: !crypto/age:Folded,NoTag |
  -----BEGIN AGE ENCRYPTED FILE-----
  YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IHNzaC1lZDI1NTE5IGYxc1ZMQSA2MWxS
  SjVLZXg3a1BqL2hWVzE2UXVHL0NmWVgxUDEwbGIrQVltaFZxR2pNClBnb1ZtSzhz
  MThTbHA0Y283bGdISnN0ZzBrMzcyM0tJOU1OQUh5Y0RmY0EKLS0tIHFFQ2pWcWNp
  Vmt0YTcwRkxhWmtSd0VINllUVVhOZXI0L1laRWJ3dkMwb2cK/xwuj6I73y3wCxQz
  wKaIkGyQNyTfscz//3hnw20fcNlI4QXyc69FxpHROi0kZ7jyFHVQYu9yilkO+MnH
  o/CGoLJ3MQ==
  -----END AGE ENCRYPTED FILE-----

$ age -R ~/.ssh/id_ed25519.pub -y test.yaml | age -i ~/.ssh/id_ed25519 -d -y
hey1: !crypto/age This is a string
hey2: &hey2 !crypto/age:SingleQuoted 'This is a single quoted string'
# This is a head comment
hey3:
  subhey: !crypto/age:DoubleQuoted "This is a double quoted string" # this is a line comment
  # This is a foot comment
hey4: !crypto/age:Literal |-
  This is a literal string
hey5: !crypto/age:Folded >-
  This is a folded string
hey6: *hey2
---
hey1: This is a string with no tag
hey2: 'This is a single quoted string with no tag'
hey3: "This is a double quoted string with no tag"
hey4: |-
  This is a literal string with no tag
hey5: >-
  This is a folded string with no tag

[sylr]

Hi @FiloSottile 👋

I am wondering if I could get your feedback on this ?

Thank you and happy holidays 🥳

[sylr]

All right, I'm pretty happy with the current state of this PR.

I believe it could be really useful for gitops.

[sylr]

Hi @FiloSottile,

I'd like to know if you interested in merging this feature.

Regards.

[wgslr]

I would be extremely surprised if Filippo was willing to merge this feature. age is so far a totally general purpose encryption tool, while this merge requests caters to a very specific and opinionated use and a single file format. Merging such additional features means additional work with maintaining them down the road, or perhaps even opens some unexpected holes.
I think a much better way forward would be for you to build a separate utility for processing yaml files that uses age for encryption under the hood.

[sylr]

@wgslr Filippo seemed to like this feature.

Merging such additional features means additional work with maintaining them down the road, or perhaps even opens some unexpected holes.

Yes, but if you follow this line you get nothing done. I made sure to have a proper test coverage in both the YAML wrapper I made for this and in this PR so that the maintaining effort is lowered to an acceptable level.

I think a much better way forward would be for you to build a separate utility for processing yaml files that uses age for encryption under the hood.

I don't see how spreading maintaining efforts across multiple projects can be beneficial to anyone.

Anyway, I must admit I'm counting on this being merged into the official project to have some leverage for getting the same kind of YAML support merged into https://github.com/kubernetes-sigs/kustomize/.

I see it as a 2 way street:

• kustomize would get a new "secure layer" that allows people to manage clear YAML manifests while securing credentials.
• age gets a broader audience and thus more potential people willing to help the maintaining effort.

[sylr]

Hi,

Since I did not get feedback on this I've decided to maintain this in its own repo and close this.

Regards.