DNSSEC algorithm roll-over
Intro
tlakh.xyz uses PowerDNS running on OpenBSD as a hidden signer. Zones
are transferred via AXFR to authoritative nameservers running NSD on
OpenBSD. Version 4.3 of PowerDNS introduced support for algorithm
roll-overs. We wanted to change the signing algorithm from RSASHA512
(Algorithm 10) to ECDSAP256SHA256 (Algorithm 13) as recommended by
RFC 8624. We followed RFC 6781 for the rollover steps.
Roll-over
The following subsections will use the state names from RFC 6781.
initial
On June 8th 2020 tlakh.xyz was signed with RSASHA512 (Algorithm 10). We wanted to change the algorithm to ECDSAP256SHA256 (Algorithm 13).
$ pdnsutil list-keys tlakh.xyz Zone Type Size Algorithm ID Location Keytag ---------------------------------------------------------------------------------- tlakh.xyz ZSK 2048 RSASHA512 6 cryptokeys 65156 tlakh.xyz KSK 4096 RSASHA512 5 cryptokeys 15216
new RRSIGs
DNSKEYs and RRSIGs in the tlakh.xyz zone have a TTL of 86000 seconds (1 day):
$ dig @a.ns.sha256.net. +dnssec +norec +noall +answer tlakh.xyz DNSKEY tlakh.xyz. 86400 IN DNSKEY 256 3 10 AwEAAaqRIYWrsASI40dwuwfbo04WT0SAKOi3espbBQuRIRS0t74isCgN H7lCzOf5AW50fwSWpceiY5CB7gddvKCJIJyBrRLkaFdT5cPGDfklNcYY Cp+pv8u1umzoiDtpoDZcnqtO7+0TuGZVweMLrVajrapZkeSp3h4I1kDw PQhcpcJnuYeN/nMtLggfX19X/sXPKo6Gm23n3gTXp8EZu9dGy5KcQYdx ilQCUL2RVJqoYBDOoLtF3spthEXbsxDobCPz2zbzENvNWLtV7aZSiefu SoBfZlGxC9eWypo5LtCaJlfQiUktFrB0BqrmIWqHxuAa2c1+bZuhdlEq 4Oa+UGd4N9M= tlakh.xyz. 86400 IN DNSKEY 257 3 10 AwEAAeoCANNycAHU3FtrctGycQ1/I5pN8iWNSZVhruxJsyiD75H7Mzet /gWRLiNmJ6e/aFPYuvWtdOjFyfOec5gIlI9J9cxY4L3KRSkeB/wjPkxf 9GXvqxcDLg3P1eaC63/rPdhjfgq3nE3Bw3NXlTuD6SWB6YdfioiyVo+e JThrYhaFqKzPqZbGn3fEGuOp39zJ+Qunq98Vg7oTh0ch3k2H9XhRP3W+ zEPnvmPKLo9+k92xvfZasgCay8vjaNRQubn9nNtNwUPKJSCIXKvmrykB PLAXBcjHlFSc6D7g4jVwzWrYtEeAA+fxqA/UBXGFrJWC3ZdD/mtDkT+v JKAL4HqCojFrRKgWq//QenhjZeZ0Efq767ZvZvqoyNweTcwGdXYteRCB R0qV4TLjD8vMczMfFboZkEJo4Xj8xDDmoslErlMGsC8TJ0uQeKB6YqKI dRJqQwtrFHx+rxFvA3+SAcKlccjZo2024f2Rq0lUSb838j1z1xY9ACh/ ht0ixk0bArQ/TdqNC6SwTniiQaJfmIik64gCZE7sxMJmryxkEjtHiLie Czls4RUMpuIc3F6d/3Gq75sgt129bYWWzNIsGaqZKL97Zl4qVpOsK3I7 yX7gNR7ogp7d/bGj11BUOU3ZsmJ30tvcD8CdNhokXFTyx8Z4QvNuUJKt TalgU/yRpCwTclRB tlakh.xyz. 86400 IN RRSIG DNSKEY 10 2 86400 20200618000000 20200528000000 15216 tlakh.xyz. gE3lNMY/Ted8nvgXH+rBm+uuKMUly5fp061Hd9kePhZSvZWzH2gaaTaJ s0kXUNBYsuUX1BeThZWPLqJDEKk9hkiffT8Mt7dBVsP9cS7rj8sM10st UWN7vrxpY4dcToknuyRaIiHU7K0/0pGWcgUcTJnwfuJfmDYexNZUf4mW kG5Ro89sSLZR3c9peKvXUig7f61e3QbS1m0h1ZsEf/hQuozb354z+x2I 0zv1LqFZt8IOTF5AD5RcZe1OatJlF02Z5Yzkj75uwa5MTD8Gfwu1vmTL 9gOieVu+10PJub7y62kcr5ZMmkUXeTHcMG+Oy6Y9IOMBMF2btNmCDY2P rGNReiRDSnQEU9726KeVGtlhyAjqDwCFuFWYug3cCJZ98aQrOXSjWXTG XyOyO+fxT2BfzUbq4L35xv34f83g5ulZvFO/oUXz1Rulhut3UUSGyev3 jqzQ4VIcYwsXRRWrlG+fZUhYtDjXCcqAtZyHtOY2oU5CNKuYDvyZMdAQ voecdB0VzzX3TXBV+ykpPeLp/qOKhxRYZao4p7ZkXqHAxSXjrV3ws2// CQhD43ex+qleMGPrlQkHa5sjwGhgvfEyqV2YKOcq41I/j8nltHoZy2sR 6NlFv1TAWlNgK4bHGQmQHTnC95URgSzFuemy4d6JDo/htFLfTIMRjWbj 9OqBoT/8xgw=
After introducing RRSIGs made with the new key we had to wait at least 1 day for the new RRset to propagate to caches. At about 16:45 UTC on June 8th 2020 we introduced new ZSK and KSK keys with algorithm 13 but set the key to unpublished:
$ pdnsutil add-zone-key tlakh.xyz zsk active unpublished ecdsa256 Added a ZSK with algorithm = 13, active=1 $ pdnsutil add-zone-key tlakh.xyz ksk active unpublished ecdsa256 Added a KSK with algorithm = 13, active=1 $ pdnsutil list-keys tlakh.xyz Zone Type Size Algorithm ID Location Keytag ---------------------------------------------------------------------------------- tlakh.xyz ZSK 2048 RSASHA512 6 cryptokeys 65156 tlakh.xyz ZSK 256 ECDSAP256SHA256 13 cryptokeys 60132 tlakh.xyz KSK 4096 RSASHA512 5 cryptokeys 15216 tlakh.xyz KSK 256 ECDSAP256SHA256 14 cryptokeys 22433
The newly signed zone had now double signatures:
$ dig @a.ns.sha256.net. +dnssec +norec +noall +answer tlakh.xyz A tlakh.xyz. 86400 IN A 45.32.179.105 tlakh.xyz. 86400 IN RRSIG A 10 2 86400 20200618000000 20200528000000 65156 tlakh.xyz. ocpnfmI2U0l24+PGUhiJwYaezqpFnpTgTphW6zfuc8uIqYrc94xcGx9o 9Bt6RoSWd1X0DG2BKWZKHI+5NEFZ1YQvTP3n5MzPNP8f9KCUkriY0Y6z RwxZJK9x/m5HuB9Nd1+sASFzc4rZme/EKGFvbGooAznFe2WAxblNLxA/ yrXHwuP5tBh4SYrgayQCFWHgrbtJfS57d/s/KorwhwQIAsiqLg68rFV3 IPjaKjWWgQfEsiAq0fuEULuRTZffqdMrLtzj9LHo2h3n9jKwHZ/B/8Cs gi3/Cu62PlBOtSRBi107jyC6TXmTzyK6YdhjJ0heam3eFXo7vSAmmTj8 UagaIQ== tlakh.xyz. 86400 IN RRSIG A 13 2 86400 20200618000000 20200528000000 60132 tlakh.xyz. Gqg0ML2H/O3EFSH1IolyrwGmbt/U6RkMxHnz7w1OGzmP+d4c7hyFuNdb 2zZXlTVYws0RnExAoY/3rOF7dTa3IA==
dnsviz (local copy) also saw RRsets from the new keys but not the keys themselves.
new DNSKEY
At about 16:55 UTC on June 9th 2020 we published the new DNSKEYs:
$ pdnsutil publish-zone-key tlakh.xyz 13 $ pdnsutil publish-zone-key tlakh.xyz 14
The newly signed zone now had 4 DNSKEYs:
$ dig @a.ns.sha256.net. +dnssec +norec +noall +answer tlakh.xyz DNSKEY tlakh.xyz. 86400 IN DNSKEY 256 3 10 AwEAAaqRIYWrsASI40dwuwfbo04WT0SAKOi3espbBQuRIRS0t74isCgN H7lCzOf5AW50fwSWpceiY5CB7gddvKCJIJyBrRLkaFdT5cPGDfklNcYY Cp+pv8u1umzoiDtpoDZcnqtO7+0TuGZVweMLrVajrapZkeSp3h4I1kDw PQhcpcJnuYeN/nMtLggfX19X/sXPKo6Gm23n3gTXp8EZu9dGy5KcQYdx ilQCUL2RVJqoYBDOoLtF3spthEXbsxDobCPz2zbzENvNWLtV7aZSiefu SoBfZlGxC9eWypo5LtCaJlfQiUktFrB0BqrmIWqHxuAa2c1+bZuhdlEq 4Oa+UGd4N9M= tlakh.xyz. 86400 IN DNSKEY 256 3 13 9Du0N8A9jI+w3gbBcuyaaL9YMM/ooAJvvpewmooIfWajgFPyLvGhr0zR ylhMCoVtbl4XSMD+di1LMiAIhPN9Eg== tlakh.xyz. 86400 IN DNSKEY 257 3 10 AwEAAeoCANNycAHU3FtrctGycQ1/I5pN8iWNSZVhruxJsyiD75H7Mzet /gWRLiNmJ6e/aFPYuvWtdOjFyfOec5gIlI9J9cxY4L3KRSkeB/wjPkxf 9GXvqxcDLg3P1eaC63/rPdhjfgq3nE3Bw3NXlTuD6SWB6YdfioiyVo+e JThrYhaFqKzPqZbGn3fEGuOp39zJ+Qunq98Vg7oTh0ch3k2H9XhRP3W+ zEPnvmPKLo9+k92xvfZasgCay8vjaNRQubn9nNtNwUPKJSCIXKvmrykB PLAXBcjHlFSc6D7g4jVwzWrYtEeAA+fxqA/UBXGFrJWC3ZdD/mtDkT+v JKAL4HqCojFrRKgWq//QenhjZeZ0Efq767ZvZvqoyNweTcwGdXYteRCB R0qV4TLjD8vMczMfFboZkEJo4Xj8xDDmoslErlMGsC8TJ0uQeKB6YqKI dRJqQwtrFHx+rxFvA3+SAcKlccjZo2024f2Rq0lUSb838j1z1xY9ACh/ ht0ixk0bArQ/TdqNC6SwTniiQaJfmIik64gCZE7sxMJmryxkEjtHiLie Czls4RUMpuIc3F6d/3Gq75sgt129bYWWzNIsGaqZKL97Zl4qVpOsK3I7 yX7gNR7ogp7d/bGj11BUOU3ZsmJ30tvcD8CdNhokXFTyx8Z4QvNuUJKt TalgU/yRpCwTclRB tlakh.xyz. 86400 IN DNSKEY 257 3 13 p3cC5drs8jRsY7um9Bb4QyEfecpyY5oXh45X4FskftvDAda46SVT9WVe uqIgofK/XfEW15ugLkWjF5uHCKPWsw== tlakh.xyz. 86400 IN RRSIG DNSKEY 10 2 86400 20200618000000 20200528000000 15216 tlakh.xyz. H5ZuAY1cMy3IPQiRahFzO4XeFpkFD1IRNSxffBL/JrfAsg3WuKEHBjhN NefYeMccydd/TB4A+D01xUKHqTEg6HhEbeCdzbihEmgHZeMXIt6G/OVA jWqEyspahK5AbDyIAWoKInaDC9NfHA8uxqFmnU7dpVg26hhuAaiQJE7j RxyoKXZY857jzEZf6E62QHw/7l9z/e5R94R/Nfc73Ch57MyWsH7pY+CS KXI6KhrfK5wY/paDPLzWP48KZ5VoP+laPFSV1qFYFa40hk/Z0wbZGQSm iKrk3Dfu6lagEeYfXDaCzISauwCYbxTw4l8adXbbBypAtdrFqcUaaCZG 5KjOFcYrUtymaucShbwjfcWrZdJTd4D32tNrWhv17QQCM1k3M7uO8FdG jyPPfoChRSh3Hd5h4v8z2bkjIrMd4Z54xeaxoL49+2R0L0ei6L/4pxap 7SVVOkqICTlT4nMI2XihTEmmqFeOQNoKdgYb/VHZqWP9n8jqlXf5emr6 UQS8bSH1pjigslY7ug8bW/tvfcPX2AtAXW2M0HmxgOlbxFC8AqYJom5l dqpPbTeyyXawE/TBf/naAvkXpzyYoIU1N5oI4ckRyEaJEO2rjgmtn4fA JDo2HjMmssFyiH/pGSSiV/ZbOqri6XecsKOIgr5LvzMeAHRkw9od2Kmg Y9NUjUfPMVk= tlakh.xyz. 86400 IN RRSIG DNSKEY 13 2 86400 20200618000000 20200528000000 22433 tlakh.xyz. rtFezrjl4R0A8SvyYCDg5M1SNASINPcLqNdYzveKqq80sVqKwmvr+o9l IQMFPE5PMIFYC7SS5utV8I5RqNV/7Q==
dnsviz (local copy) also saw all DNSKEYs. There seemed to be arrows missing from the DNSKEY with id 22433 to all other DNSKEYs on the dnsviz visualisation but the responses showed the RRSIG from the new DNSKEY.
new DS
The old DNSKYE RRset containing only two keys expired around 17:00 UTC on June 10th 2020. At this point the old DS record could have been replaced with a new DS record but we didn't get around to it.
We continued on the morning of the 11th, first we fetched the NS set for xyz:
$ dig +noall +answer xyz NS xyz. 37933 IN NS x.nic.xyz. xyz. 37933 IN NS y.nic.xyz. xyz. 37933 IN NS z.nic.xyz. xyz. 37933 IN NS generationxyz.nic.xyz.
and checked all of them for the DS record for tlakh.xyz:
$ for i in x y z generationxyz; do dig @$i.nic.xyz +noall +answer tlakh.xyz DS; done tlakh.xyz. 3600 IN DS 15216 10 2 38C610B933677FC6BB5E39E3649646691AABDD5100D0BF6362E5A095 4477CC8A tlakh.xyz. 3600 IN DS 15216 10 2 38C610B933677FC6BB5E39E3649646691AABDD5100D0BF6362E5A095 4477CC8A tlakh.xyz. 3600 IN DS 15216 10 2 38C610B933677FC6BB5E39E3649646691AABDD5100D0BF6362E5A095 4477CC8A tlakh.xyz. 3600 IN DS 15216 10 2 38C610B933677FC6BB5E39E3649646691AABDD5100D0BF6362E5A095 4477CC8A
The TTL was 3600 seconds or one hour. After introducing the new DS record we had to wait at least this long. This is complicated by the fact that we did not now how long it would take for the DS record to show up in the xyz zone and how long it would take for the xyz zone to propagate to all authoritative nameservers. xyz probably employed anycast as well so it would be very difficult for us to observe all nameservers. The registrar for tlakh.xyz wants the DNSKEY to submit the DS record to the registry:
$ pdnsutil export-zone-dnskey tlakh.xyz 14 tlakh.xyz IN DNSKEY 257 3 13 p3cC5drs8jRsY7um9Bb4QyEfecpyY5oXh45X4FskftvDAda46SVT9WVeuqIgofK/XfEW15ugLkWjF5uHCKPWsw==
We entered the DNSKEY in the registrar webinterface at about 05:45 UTC on June 11th 2020. At 05:52 we started to see the new DS record on some authoritative nameservers:
$ for i in x y z generationxyz; do dig @$i.nic.xyz +noall +answer tlakh.xyz DS; done tlakh.xyz. 3600 IN DS 15216 10 2 38C610B933677FC6BB5E39E3649646691AABDD5100D0BF6362E5A095 4477CC8A tlakh.xyz. 3600 IN DS 22433 13 2 692C34230671F2CD2A2D7DC7432B373B556D357787883DE754660A69 E4F6D05F tlakh.xyz. 3600 IN DS 22433 13 2 692C34230671F2CD2A2D7DC7432B373B556D357787883DE754660A69 E4F6D05F tlakh.xyz. 3600 IN DS 15216 10 2 38C610B933677FC6BB5E39E3649646691AABDD5100D0BF6362E5A095 4477CC8A
And at 05:53 it was visible on all nameservers, at least from this vantage point:
$ for i in x y z generationxyz; do dig @$i.nic.xyz +noall +answer +norec tlakh.xyz DS; done tlakh.xyz. 3600 IN DS 22433 13 2 692C34230671F2CD2A2D7DC7432B373B556D357787883DE754660A69 E4F6D05F tlakh.xyz. 3600 IN DS 22433 13 2 692C34230671F2CD2A2D7DC7432B373B556D357787883DE754660A69 E4F6D05F tlakh.xyz. 3600 IN DS 22433 13 2 692C34230671F2CD2A2D7DC7432B373B556D357787883DE754660A69 E4F6D05F tlakh.xyz. 3600 IN DS 22433 13 2 692C34230671F2CD2A2D7DC7432B373B556D357787883DE754660A69 E4F6D05F
We then setup 4 RIPE Atlas measurements to query x.nic.xyz, y.nic.xyz, z.nic.xyz, and generationxyz.nic.xyz for the DS record of tlakh.xyz from 500 probes world wide. We used the RIPE Atlas cli tool to analyse the results, for example for x.nic.xyz:
$ ripe-atlas report 25704650 | fgrep DS | sort | uniq -c 468 ;tlakh.xyz. IN DS 1 tlakh.xyz. 3577 IN DS 22433 13 2 692c34230671f2cd2a2d7dc7432b373b556d357787883de754660a69e4f6d05f 457 tlakh.xyz. 3600 IN DS 22433 13 2 692c34230671f2cd2a2d7dc7432b373b556d357787883de754660a69e4f6d05f
At about 06:00 UTC we were confident that the new DS record had propagated world wide. dnsviz (local copy) saw the new DS record pointing to the new DNSKEY.
DNSKEY removal
With a TTL of one hour for the DS record in the xyz zone the old DNSKEYs could have been removed at 07:00 UTC on June 11th 2020. We removed them one hour later, at around 08:00 UTC:
$ pdnsutil unpublish-zone-key tlakh.xyz 6 $ pdnsutil unpublish-zone-key tlakh.xyz 5
We saw two DNSKEYs instead of four, one ZSK and one KSK. There are still two RRSIGs, one with algorithm 10, the old one and one with algorithm 13 since we only unpublished the old keys but they are still used for signing.
$ dig @a.ns.sha256.net. +dnssec +norec +noall +answer tlakh.xyz DNSKEY tlakh.xyz. 86400 IN DNSKEY 256 3 13 9Du0N8A9jI+w3gbBcuyaaL9YMM/ooAJvvpewmooIfWajgFPyLvGhr0zR ylhMCoVtbl4XSMD+di1LMiAIhPN9Eg== tlakh.xyz. 86400 IN DNSKEY 257 3 13 p3cC5drs8jRsY7um9Bb4QyEfecpyY5oXh45X4FskftvDAda46SVT9WVe uqIgofK/XfEW15ugLkWjF5uHCKPWsw== tlakh.xyz. 86400 IN RRSIG DNSKEY 10 2 86400 20200625000000 20200604000000 15216 tlakh.xyz. dGiAsG2KyIgivCEsEwXpCUg8vHspOJcDavDWF4ob5D4AaGxOg2rsUDeu AhbNRfKjWVwNOYNf4zxyqqDNAQeyU00ZsrBDhWkz4gGH8MHddB8quLzX vQDjhv4gHepidFOy1QIyKGsgvwPoxSDf5VpHYJxUiZKSq1AERT/IeR0Q DOqQcJ/UAjRLdXDox3JqFnwmvXoyY5SDjxIoHiRU5gnmEmDpFyvrLMUY SfQ8LvU4KV0UFIPWHjyApgysk2YFJfLWFiKrBZMAaD/aD6rTuvPIdPq7 AYGC5YpsR3+6m6S1uKKfeC2ZdnacKdVgVWcIuL3KrnHflpSGtEcL7Y2V gBYY96eEWKSQ8IlOp5fpIQbQcw31R3dQeQWuac8U3NmH+X2UNzZtozLI 5d5U8ZqYZKkoVh6K4cCxZjkx6UnFeSL4FAHxFc05/1sixED8ueFhCa15 NdKaRpPJXjTAqN5Ans8Z/jJ+aosg7Cnn7BrQTg0/qhU9TYY7U8PdaBaP cCwrkENdA1LvitXa/kI9G2r9c0WMkUh5zOcVxytmYot5zVzjXoB7lD2q OE0efBCKEfwI93aOD3CdS/9aE6eB7A0TAhI/MejCho3dNFM48TF57EHc 1WOVxFVdAYyw8bcKTFV5sQOzt5YN3iU4TD4Y3ZnJ0w3/LhD24ngm/A3p zNmtB2mqwhI= tlakh.xyz. 86400 IN RRSIG DNSKEY 13 2 86400 20200625000000 20200604000000 22433 tlakh.xyz. CbBp81aMiMLXoAbJuPA0XymHiYJGyWiIKXxAQpoTWGN7sc2P/mF/Ea9V Rg1tYw392vEhM/bi9GjHHnzNQR6+1g==
dnsviz (local copy) saw RRSIGs from the old key but no longer the key.
RRSIGs removal
One day later, at about 08:15 on June 12th 2020 it was time to completely remove the old keys:
$ pdnsutil remove-zone-key tlakh.xyz 5 $ pdnsutil remove-zone-key tlakh.xyz 6
The DNSKYEY RRset was only signed by the new and now only KSK key:
$ dig @a.ns.sha256.net. +dnssec +norec +noall +answer tlakh.xyz DNSKEY tlakh.xyz. 86400 IN DNSKEY 256 3 13 9Du0N8A9jI+w3gbBcuyaaL9YMM/ooAJvvpewmooIfWajgFPyLvGhr0zR ylhMCoVtbl4XSMD+di1LMiAIhPN9Eg== tlakh.xyz. 86400 IN DNSKEY 257 3 13 p3cC5drs8jRsY7um9Bb4QyEfecpyY5oXh45X4FskftvDAda46SVT9WVe uqIgofK/XfEW15ugLkWjF5uHCKPWsw== tlakh.xyz. 86400 IN RRSIG DNSKEY 13 2 86400 20200625000000 20200604000000 22433 tlakh.xyz. CbBp81aMiMLXoAbJuPA0XymHiYJGyWiIKXxAQpoTWGN7sc2P/mF/Ea9V Rg1tYw392vEhM/bi9GjHHnzNQR6+1g==
dnsviz (local copy) confirmed this. With this the algorithm roll-over was done.
