[Pdns-dev] Schema change practices in pdns

[Pieter Lexis]

Hi Atalay,

On 10/18/19 9:24 PM, Atalay Karatay wrote:
> Our main focus here is to understand the co-evolution/change-coupling of
> software artifacts, especially in the case of database-related changes.
>
> Since there are hardly works or established tools considering the
> co-evolution of schema and the code accessing it, we are investigating
> this topic.

Neat! I'm looking forward to the report.

> As powerdns/pdns is one major project in the open-source-community and
> is also supporting several backend systems, I have some questions:
>
> We found out that schema related changes were often committed together
> - across all databases .
>
> So in your case for instance, some schema changes were done for mysql,
> postgresql, sqlite, etc.  simultaneously.
>
> 1) Could you describe the common process of introducing these
> schema changes or do you have a manual for that?

Schema-changes are added to PowerDNS to either support new features in
PowerDNS, e.g. when Disabled Data and Comments were added in 3.4.0[1].
Another reason to do schema changes is fixing bugs due to the existing
schema, e.g. because a wrong type was used[2] or newer database versions
have different defaults.

We will usually follow the latest releases of the backend databases we
support when issues are found, where we take care not to break things
for users running newer PowerDNS on distros with older databases. But
all of this is a manual process.

> 2) Is there any tool in use, which supports this process automatically
> or is it done manually?

The schemas and upgrade SQL-files are artisinally handcrafted :). Apart
from the LMDB backend, PowerDNS does not maintain the schema in the
backend database. Combined with the ability to override any query via
the configuration, an administrator can use their own schema or use
PowerDNS on top of a database that was used for zone-data before
switching to PowerDNS.

Although we do recommend using the provided schema as that schema is
battle-tested.

> 3) How much time is consumed when introducing schema changes in your
> project?

It depends, "However long it takes to write". It is usually relatively
clear what changes need to happen before the schema is changed. After
which the upgrade SQL statements need to be written and tested.
All-in-all I'd figure a good days' work.

> 4) Which problems occur when changing the schema?

Hopefully it fixes problems! The biggest problems that occur are users
complaining that they upgraded the software without also upgrading the
schema.

If you want to know more, don't hesitate to reply to the list.

Best regards,

Pieter

1 -
https://doc.powerdns.com/authoritative/changelog/pre-4.0.html#powerdns-authoritative-server-3-4-0
2 - https://github.com/PowerDNS/pdns/pull/5225

-- 
Pieter Lexis
PowerDNS.COM BV -- https://www.powerdns.com