dnscontrol/.github/workflows/pr_integration_tests.yml
Jakob Ackermann 1e67585e8f
HETZNER_V2: Add provider for Hetzner DNS API (#3837)
Closes https://github.com/StackExchange/dnscontrol/issues/3787

This PR is adding a `HETZNER_V2` provider for the "new" Hetzner DNS API.

Testing:
- The integration tests are passing.
- Manual testing:
  - `preview` (see diff for existing zone)
- `preview --populate-on-preview` (see full diff for newly created zone)
  - `push` (see full diff; no diff after push)
- `push` (see full diff; no diff after push to newly created zone --
i.e. single pass and done)

```js
var REG_NONE = NewRegistrar('none')
var DSP = NewDnsProvider('HETZNER_V2')

D('testing-2025-11-14-7.dev', REG_NONE, DnsProvider(DSP),
    A('@', '127.0.0.1')
)
```

<details>

```
# push for newly created zone
CONCURRENTLY checking for 1 zone(s)
SERIALLY checking for 0 zone(s)
Waiting for concurrent checking(s) to complete...DONE
******************** Domain: testing-2025-11-14-7.dev
1 correction (HETZNER_V2)
#1: Ensuring zone "testing-2025-11-14-7.dev" exists in "HETZNER_V2"
SUCCESS!
CONCURRENTLY gathering records of 1 zone(s)
SERIALLY gathering records of 0 zone(s)
Waiting for concurrent gathering(s) to complete...DONE
******************** Domain: testing-2025-11-14-7.dev
4 corrections (HETZNER_V2)
#1: ± MODIFY-TTL testing-2025-11-14-7.dev NS helium.ns.hetzner.de. ttl=(3600->300)
± MODIFY-TTL testing-2025-11-14-7.dev NS hydrogen.ns.hetzner.com. ttl=(3600->300)
± MODIFY-TTL testing-2025-11-14-7.dev NS oxygen.ns.hetzner.com. ttl=(3600->300)
SUCCESS!
#2: + CREATE testing-2025-11-14-7.dev A 127.0.0.1 ttl=300
SUCCESS!
Done. 5 corrections.
```
</details>

Feedback for @jooola and @LKaemmerling:
- The SDK was very useful in getting 80% there! Nice! 🎉 
- Footgun:
- The `result` values are not "up-to-date" after waiting for an
`Action`, e.g. `Zone.AuthoritativeNameservers.Assigned` is not set when
`Client.Zone.Create()` returns and the following "wait" will not update
it.
- Taking a step back here: Waiting for an `Action` with a separate SDK
call does not seem very natural to me. Does the SDK-user need to know
that you are processing operations asynchronous? (Which seems like an
implementation detail to me, something that the SDK could abstrct over.)
Can `Client.Zone.Create()` return the final `Zone` instead of the
intermediate result?
- Features missing compared to the DNS Console, in priority order:
- It is no longer possible to remove your provided name servers from the
root/apex. Use-case: dual-home/multi-home zone with fewer than three
servers from Hetzner. I'm operating one of these and cannot migrate over
until this is fixed.
- Performance regression due to lack of bulk create/modify. E.g. [one of
the test
suites](a71b89e5a2/integrationTest/integration_test.go (L619))
spends about 4.5 minutes on making creating 100 record-sets and then
another 4 minutes for deleting them in sequence again. With your async
API, these are `create 2*100 + delete 2*100 = 400` API calls.
Previously, these were `create 1 + delete 100 = 101` API calls. Are you
planning on adding batch processing again?
- Usability nits
- Compared to other record-set based APIs, upserts for record-sets are
missing. This applies to records of a record-set and the ttl of the
record-set (see separate SDK calls for the cases `diff2.CREATE` vs
`diff2.CHANGE` and two calls in `diff2.CHANGE` for updating the TTL vs
records).
- Some SDK methods return an `Action` (e.g. `Zone.ChangeRRSetTTL()`),
others wrap the `Action` in a struct (`Client.Zone.CreateRRSet()`) --
even when the struct has a single field (`ZoneRRSetDeleteResult`).

---------

Co-authored-by: "Jonas L." <jooola@users.noreply.github.com>
Co-authored-by: "Lukas Kämmerling" <LKaemmerling@users.noreply.github.com>
Co-authored-by: Tom Limoncelli <6293917+tlimoncelli@users.noreply.github.com>
2025-11-30 09:14:54 -05:00

220 lines
9.5 KiB
YAML

name: "PR: Run INTEGRATION tests"
# When will this pipeline be activated?
# 1. On any pull-request, or if someone pushes to a branch called
# "tlim_testpr".
on:
pull_request:
workflow_dispatch:
# Want to trigger all the tests without making a PR?
# Run: git push origin main:tlim_testpr --force
# This will trigger a full PR test on the main branch.
# See https://github.com/StackExchange/dnscontrol/actions/workflows/pr_integration_tests.yml?query=branch%3Atlim_testpr
push:
branches:
- 'tlim_testpr'
# Environment Variables
env:
# cache-key: Change to force cache reset `pwsh > Get-Date -UFormat %s`
cache-key: 1639697695
# go-mod-path: Where go-mod writes temp files
go-mod-path: /go/pkg/mod
# BIND_DOMAIN: BIND is the one providers that we always test. By
# defining this here, we know it will always be set.
BIND_DOMAIN: example.com
jobs:
# integration-test-providers: Determine which providers have a _DOMAIN variable set.
# That variable enables testing for the provider. The results are
# stored in a JSON blob stashed in # needs.integration-test-providers.outputs.integration_test_providers
# where integration-tests can pick it up.
integration-test-providers:
#needs: build
runs-on: ubuntu-latest
outputs:
integration_test_providers: ${{ steps.get_integration_test_providers.outputs.integration_test_providers }}
steps:
- name: Set Integration Test Providers
id: get_integration_test_providers
shell: pwsh
run: |
$Providers = @()
$EnvContext = ConvertFrom-Json -InputObject $env:ENV_CONTEXT
$VarsContext = ConvertFrom-Json -InputObject $env:VARS_CONTEXT
$SecretsContext = ConvertFrom-Json -InputObject $env:SECRETS_CONTEXT
ConvertFrom-Json -InputObject $env:PROVIDERS | ForEach-Object {
if(($null -ne $EnvContext."$($_)_DOMAIN") -or ($null -ne $VarsContext."$($_)_DOMAIN") -or ($null -ne $SecretsContext."$($_)_DOMAIN")) {
$Providers += $_
}
}
Write-Host "Integration test providers: $Providers"
echo "integration_test_providers=$(ConvertTo-Json -InputObject $Providers -Compress)" >> $env:GITHUB_OUTPUT
env:
PROVIDERS: "['AXFRDDNS', 'AXFRDDNS_DNSSEC', 'AZURE_DNS','BIND','BUNNY_DNS','CLOUDFLAREAPI','CLOUDNS','CNR','DIGITALOCEAN','FORTIGATE','GANDI_V5','GCLOUD','HEDNS','HETZNER_V2','HEXONET','HUAWEICLOUD','INWX','JOKER','MYTHICBEASTS', 'NAMEDOTCOM','NS1','POWERDNS','ROUTE53','SAKURACLOUD','TRANSIP']"
ENV_CONTEXT: ${{ toJson(env) }}
VARS_CONTEXT: ${{ toJson(vars) }}
SECRETS_CONTEXT: ${{ toJson(secrets) }}
# integration-tests: Run the integration tests on any provider listed
# in needs.integration-test-providers.outputs.integration_test_providers.
integration-tests:
if: github.ref != 'refs/heads/master' && github.ref != 'refs/heads/main'
runs-on: ubuntu-latest
container:
image: golang:1.25
needs:
- integration-test-providers
env:
TEST_RESULTS: "/tmp/test-results"
GOTESTSUM_FORMAT: testname
# PROVIDER DOMAIN LIST
# These providers will be tested if the env variable is set.
# Set it to the domain name to use during the test.
AXFRDDNS_DOMAIN: ${{ vars.AXFRDDNS_DOMAIN }}
AXFRDDNS_DNSSEC_DOMAIN: ${{ vars.AXFRDDNS_DNSSEC_DOMAIN }}
AZURE_DNS_DOMAIN: ${{ vars.AZURE_DNS_DOMAIN }}
BIND_DOMAIN: ${{ vars.BIND_DOMAIN }}
BUNNY_DNS_DOMAIN: ${{ vars.BUNNY_DNS_DOMAIN }}
CLOUDFLAREAPI_DOMAIN: ${{ vars.CLOUDFLAREAPI_DOMAIN }}
CLOUDNS_DOMAIN: ${{ vars.CLOUDNS_DOMAIN }}
CNR_DOMAIN: ${{ vars.CNR_DOMAIN }}
CSCGLOBAL_DOMAIN: ${{ vars.CSCGLOBAL_DOMAIN }}
DIGITALOCEAN_DOMAIN: ${{ vars.DIGITALOCEAN_DOMAIN }}
FORTIGATE_DOMAIN: ${{ vars.FORTIGATE_DOMAIN }}
GANDI_V5_DOMAIN: ${{ vars.GANDI_V5_DOMAIN }}
GCLOUD_DOMAIN: ${{ vars.GCLOUD_DOMAIN }}
HEDNS_DOMAIN: ${{ vars.HEDNS_DOMAIN }}
HETZNER_V2_DOMAIN: ${{ vars.HETZNER_V2_DOMAIN }}
HEXONET_DOMAIN: ${{ vars.HEXONET_DOMAIN }}
HUAWEICLOUD_DOMAIN: ${{ vars.HUAWEICLOUD_DOMAIN }}
JOKER_DOMAIN: ${{ vars.JOKER_DOMAIN }}
MYTHICBEASTS_DOMAIN: ${{ vars.MYTHICBEASTS_DOMAIN }}
NAMEDOTCOM_DOMAIN: ${{ vars.NAMEDOTCOM_DOMAIN }}
NS1_DOMAIN: ${{ vars.NS1_DOMAIN }}
POWERDNS_DOMAIN: ${{ vars.POWERDNS_DOMAIN }}
ROUTE53_DOMAIN: ${{ vars.ROUTE53_DOMAIN }}
SAKURACLOUD_DOMAIN: ${{ vars.SAKURACLOUD_DOMAIN }}
TRANSIP_DOMAIN: ${{ vars.TRANSIP_DOMAIN }}
# PROVIDER SECRET LIST
# The above providers have additional env variables they
# need for credentials and such.
#
AXFRDDNS_MASTER: ${{ secrets.AXFRDDNS_MASTER }}
AXFRDDNS_NAMESERVERS: ${{ secrets.AXFRDDNS_NAMESERVERS }}
AXFRDDNS_TRANSFER_KEY: ${{ secrets.AXFRDDNS_TRANSFER_KEY }}
AXFRDDNS_TRANSFER_MODE: ${{ secrets.AXFRDDNS_TRANSFER_MODE }}
AXFRDDNS_UPDATE_KEY: ${{ secrets.AXFRDDNS_UPDATE_KEY }}
AXFRDDNS_UPDATE_MODE: ${{ secrets.AXFRDDNS_UPDATE_MODE }}
#
AXFRDDNS_DNSSEC_MASTER: ${{ secrets.AXFRDDNS_DNSSEC_MASTER }}
AXFRDDNS_DNSSEC_NAMESERVERS: ${{ secrets.AXFRDDNS_DNSSEC_NAMESERVERS }}
AXFRDDNS_DNSSEC_TRANSFER_KEY: ${{ secrets.AXFRDDNS_DNSSEC_TRANSFER_KEY }}
AXFRDDNS_DNSSEC_TRANSFER_MODE: ${{ secrets.AXFRDDNS_DNSSEC_TRANSFER_MODE }}
AXFRDDNS_DNSSEC_UPDATE_KEY: ${{ secrets.AXFRDDNS_DNSSEC_UPDATE_KEY }}
AXFRDDNS_DNSSEC_UPDATE_MODE: ${{ secrets.AXFRDDNS_DNSSEC_UPDATE_MODE }}
#
AZURE_DNS_CLIENT_ID: ${{ secrets.AZURE_DNS_CLIENT_ID }}
AZURE_DNS_CLIENT_SECRET: ${{ secrets.AZURE_DNS_CLIENT_SECRET }}
AZURE_DNS_RESOURCE_GROUP: ${{ secrets.AZURE_DNS_RESOURCE_GROUP }}
AZURE_DNS_SUBSCRIPTION_ID: ${{ secrets.AZURE_DNS_SUBSCRIPTION_ID }}
AZURE_DNS_TENANT_ID: ${{ secrets.AZURE_DNS_TENANT_ID }}
#
BUNNY_DNS_API_KEY: ${{ secrets.BUNNY_DNS_API_KEY }}
#
CLOUDFLAREAPI_ACCOUNTID: ${{ secrets.CLOUDFLAREAPI_ACCOUNTID }}
CLOUDFLAREAPI_TOKEN: ${{ secrets.CLOUDFLAREAPI_TOKEN }}
#
CLOUDNS_AUTH_ID: ${{ secrets.CLOUDNS_AUTH_ID }}
CLOUDNS_AUTH_PASSWORD: ${{ secrets.CLOUDNS_AUTH_PASSWORD }}
#
CSCGLOBAL_APIKEY: ${{ secrets.CSCGLOBAL_APIKEY }}
CSCGLOBAL_USERTOKEN: ${{ secrets.CSCGLOBAL_USERTOKEN }}
#
CNR_UID: ${{ secrets.CNR_UID }}
CNR_PW: ${{ secrets.CNR_PW }}
CNR_ENTITY: ${{ secrets.CNR_ENTITY }}
#
DIGITALOCEAN_TOKEN: ${{ secrets.DIGITALOCEAN_TOKEN }}
#
FORTIGATE_API_KEY: ${{ secrets.FORTIGATE_API_KEY }}
FORTIGATE_VDOM: ${{ secrets.FORTIGATE_VDOM }}
FORTIGATE_HOST: ${{ secrets.FORTIGATE_HOST }}
#
GANDI_V5_APIKEY: ${{ secrets.GANDI_V5_APIKEY }}
#
GCLOUD_EMAIL: ${{ secrets.GCLOUD_EMAIL }}
GCLOUD_PRIVATEKEY: ${{ secrets.GCLOUD_PRIVATEKEY }}
GCLOUD_PROJECT: ${{ secrets.GCLOUD_PROJECT }}
GCLOUD_TYPE: ${{ secrets.GCLOUD_TYPE }}
#
HEDNS_PASSWORD: ${{ secrets.HEDNS_PASSWORD }}
HEDNS_TOTP_SECRET: ${{ secrets.HEDNS_TOTP_SECRET }}
HEDNS_USERNAME: ${{ secrets.HEDNS_USERNAME }}
#
HETZNER_V2_API_TOKEN: ${{ secrets.HETZNER_V2_API_TOKEN }}
#
HEXONET_ENTITY: ${{ secrets.HEXONET_ENTITY }}
HEXONET_PW: ${{ secrets.HEXONET_PW }}
HEXONET_UID: ${{ secrets.HEXONET_UID }}
#
HUAWEICLOUD_REGION: ${{ secrets.HUAWEICLOUD_REGION }}
HUAWEICLOUD_KEY_ID: ${{ secrets.HUAWEICLOUD_KEY_ID }}
HUAWEICLOUD_KEY: ${{ secrets.HUAWEICLOUD_KEY }}
#
JOKER_USERNAME: ${{ secrets.JOKER_USERNAME }}
JOKER_PASSWORD: ${{ secrets.JOKER_PASSWORD }}
#
MYTHICBEASTS_KEYID: ${{ secrets.MYTHICBEASTS_KEYID }}
MYTHICBEASTS_SECRET: ${{ secrets.MYTHICBEASTS_SECRET }}
#
NAMEDOTCOM_KEY: ${{ secrets.NAMEDOTCOM_KEY }}
NAMEDOTCOM_URL: ${{ secrets.NAMEDOTCOM_URL }}
NAMEDOTCOM_USER: ${{ secrets.NAMEDOTCOM_USER }}
#
NS1_TOKEN: ${{ secrets.NS1_TOKEN }}
#
POWERDNS_APIKEY: ${{ secrets.POWERDNS_APIKEY }}
POWERDNS_APIURL: ${{ secrets.POWERDNS_APIURL }}
POWERDNS_SERVERNAME: ${{ secrets.POWERDNS_SERVERNAME }}
#
ROUTE53_KEY: ${{ secrets.ROUTE53_KEY }}
ROUTE53_KEY_ID: ${{ secrets.ROUTE53_KEY_ID }}
#
SAKURACLOUD_ACCESS_TOKEN: ${{ secrets.SAKURACLOUD_ACCESS_TOKEN }}
SAKURACLOUD_ACCESS_TOKEN_SECRET: ${{ secrets.SAKURACLOUD_ACCESS_TOKEN_SECRET }}
#
TRANSIP_ACCOUNT_NAME: ${{ secrets.TRANSIP_ACCOUNT_NAME }}
TRANSIP_PRIVATE_KEY: ${{ secrets.TRANSIP_PRIVATE_KEY }}
concurrency:
group: ${{ github.workflow }}-${{ matrix.provider }}
strategy:
fail-fast: false
matrix:
provider: ${{ fromJson(needs.integration-test-providers.outputs.integration_test_providers )}}
steps:
- uses: actions/checkout@v6
- run: mkdir -p "$TEST_RESULTS"
- name: restore_cache
uses: actions/cache@v4.3.0
with:
key: linux-go-${{ hashFiles('go.sum') }}-${{ env.cache-key }}
restore-keys: linux-go-${{ hashFiles('go.sum') }}-${{ env.cache-key }}
path: ${{ env.go-mod-path }}
- name: Run integration tests for ${{ matrix.provider }} provider
run: |-
go install gotest.tools/gotestsum@latest
if [ -n "$${{ matrix.provider }}_DOMAIN" ] ; then
gotestsum --junitfile ${TEST_RESULTS}/gotestsum-report.xml -- -timeout 30m -v -verbose -profile ${{ matrix.provider }} -cfworkers=false
else
echo "Skip test for ${{ matrix.provider }} provider"
fi
working-directory: integrationTest
- uses: actions/upload-artifact@v5.0.0
with:
name: integration-tests-${{ matrix.provider }}
path: ${{ env.TEST_RESULTS }}