[DPE-10780] Configure PostgreSQL workload OOM protection

[oleksii-buhaiov]

Issue

DPE-10780: The charm does not configure OOM protection for the PostgreSQL
workload, leaving database availability exposed to process termination
under memory pressure.

Solution

Configure snapd's resilience.vitality-hint for charmed-postgresql
before snap installation or refresh. Preserve existing entries and their
order, append PostgreSQL only when absent, and verify the configuration
after writing it.

On fresh deployments, snapd starts Patroni with a negative OOM adjustment,
which its PostgreSQL processes inherit. This change does not force a
restart of running services. During a snap-changing refresh, the normal
snap service restart activates the setting.

A charm-only upgrade with an unchanged snap revision does not currently
run this configuration step. Activation for that upgrade path is outside
this change.

Manual validation

Use an amd64 build environment with the repository's build dependencies
installed and a working LXD Juju controller capable of creating VMs.
Use a PR checkout under your home directory that the Juju snap can access.

From the PR checkout:

charmcraftcache pack
juju add-model oom-validation

juju deploy ./postgresql_ubuntu@24.04-amd64.charm postgresql \
  --base ubuntu@24.04 \
  --constraints 'arch=amd64 virt-type=virtual-machine cores=2 mem=2G root-disk=16G' \
  --config profile=testing

Wait for the unit to become active and idle, then connect:

juju wait-for unit postgresql/0 --timeout=30m \
  --query='workload-status=="active" && agent-status=="idle"'

juju ssh postgresql/0

Inside the VM:

sudo snap get system resilience.vitality-hint

systemctl show snap.charmed-postgresql.patroni.service \
  --property=OOMScoreAdjust

group=$(systemctl show snap.charmed-postgresql.patroni.service \
  --property=ControlGroup --value)

printf 'OOM_ADJ PID COMMAND\n'
for pid in $(cat "/sys/fs/cgroup${group}/cgroup.procs"); do
    printf '%s ' "$(cat "/proc/$pid/oom_score_adj")"
    ps -p "$pid" -o pid=,args=
done

Expected on this fresh VM: the hint is charmed-postgresql, systemd
reports OOMScoreAdjust=-899, and Patroni, the PostgreSQL postmaster,
and its listed children all show -899. This value assumes PostgreSQL
is the first and only hint entry.

This checks fresh-install configuration and live process adjustments;
it does not test OOM survival or upgrade activation.

[dragomirp]

Can we use the charmlibs-snap to set and get the config?

[oleksii-buhaiov]

looks like we can, will take a look at it

[marceloneppel]

Great work, @oleksii-buhaiov! I left a comment about one important check.

[marceloneppel]

Could you also test whether a refresh from, for example, the charm revision from the 16/stable channel will enable this OOM protection when refreshed to the revision build from this PR's code?