[dnsdist] DoH issues after 1.8.3 -> 1.9.0 upgrade

[Remi Gacogne]

Hi,

On 18/03/2024 22:00, Christoph via dnsdist wrote:
>> This might be related:https://github.com/PowerDNS/pdns/issues/13850, 
>> not backported yet
> 
> thanks for the pointer, really looking forward to the dnsdist version
> that has this solved.

Sure, I expect to release 1.9.2 including this fix in the next couple weeks.

>> The new nghttp2 provider for
>> incoming DNS over HTTPS does not support HTTP/1.1. In 1.9.x it's
>> still possible to switch back to the legacy h2o provider but note
>> that it will likely go away in the next major version of DNSdist. In
>> our testing the lack of HTTP/1.1 support was not an issue for actual
>> DNS over HTTPS clients, with most of HTTP/1.1 queries coming from
>> crawlers/bots, but of course we will reconsider if you find out that
>> legitimate DoH clients are impacted.
> 
> we see about 5-10% of non-version 2 DoH requests by looking at:
> 
> sum by (version)
> (irate(dnsdist_frontend_doh_http_version_queries{job="$job"}[$__rate_interval]))

Note that this metric (doh_http_version_queries) is incremented after 
doing some sanity checks but before actually parsing the DNS query, so 
unfortunately we cannot be sure these are valid DoH queries. At this 
point they could be bots. Can you check doh_version_status_responses for 
httpversion=1 and status=200 instead?

> So the practical solution to use dnsdist 1.9.0 with nghttp2 and
> still support HTTP/1.1 clients is to use a webserver like nginx in front 
> of dnsdist?

Yes, a reverse proxy like nginx or HAProxy might be the best option to 
keep HTTP/1.1 support at this point.

> I expected an increase of this metric during our partial outage but
> this value did not increase, is this expected?
> 
> irate(dnsdist_frontend_doh_version_status_responses{httpversion="1",status="400",job="$job"}[$__rate_interval])
> 
> dnsdist_frontend_noncompliantqueries also didn't increase.
> Which value is expected to increase?

I'm afraid we are currently not increasing any counter in this exact 
case, I'll see what I can do about it.

> btw:
> dnsdist's v1.9.0 answer to HTTP requests not using HTTP/2:
> 
>> This server implements RFC 8484 - DNS Queries over HTTP, and
>> requires HTTP/2 in accordance with section 5.2 of the RFC.
> 
> but RFC8484 does not actually require HTTP/2, right?
> 
> https://www.rfc-editor.org/rfc/rfc8484.html#section-5.2
>  > 5.2.  HTTP/2
>>
>> HTTP/2 [RFC7540] is the minimum RECOMMENDED version of HTTP for use 
>> with DoH.
> 
> It is recommended but not a "MUST".

You are correct, but in practice I am yet to see a DoH client using 
HTTP/1.1 in production. Bind 9, Unbound and Knot also only support DNS 
over HTTP/2. That being said, I'm really open to implementing DNS over 
HTTP/1.1 if it serves a real purpose, I just don't want to increase the 
code complexity and attack surface just to reply to crawlers..

Best regards,
-- 
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.powerdns.com/pipermail/dnsdist/attachments/20240319/124eefb8/attachment.sig>