Skip to content
This repository was archived by the owner on Jan 30, 2020. It is now read-only.

functional: close DBUS connections after running systemd commands - #1706

Merged
dongsupark merged 1 commit into
coreos:masterfrom
endocode:dongsu/dbus-conn-close-fxtests
Nov 16, 2016
Merged

functional: close DBUS connections after running systemd commands#1706
dongsupark merged 1 commit into
coreos:masterfrom
endocode:dongsu/dbus-conn-close-fxtests

Conversation

@dongsupark

Copy link
Copy Markdown

As described in #1704, functional test has a bug hitting an upper limit of ~40 tests in total. That's actually an issue of DBUS connections remaining opened even after nspawn containers got successfully terminated. That's why the number of unix sockets grows up to 256.

$ netstat -nap | grep /var/run/dbus/system_bus_socket | wc -l
256

From that moment on, functional test hangs mysteriously. Sometimes users could see errors like "The maximum number of active connections for UID 0 has been reached."

Its reason is that every DBUS connection was never closed. The more tests we add, the more stale DBUS connections we have. This bug has existed since the beginning.

Fix it by adding conn.Close() after running systemd commands.

As described in coreos#1704, functional
test has a bug hitting an upper limit of ~40 tests in total. That's
actually an issue of DBUS connections remaining opened even after
nspawn containers got successfully terminated. That's why the number
of unix sockets grows up to 256.

  $ netstat -nap | grep /var/run/dbus/system_bus_socket | wc -l
  256

From that moment on, functional test hangs mysteriously. Sometimes
users could see errors like "The maximum number of active connections
for UID 0 has been reached."

Its reason is that every DBUS connection was never closed. The more
tests we add, the more stale DBUS connections we have. This bug has
existed since the beginning.

Fix it by adding conn.Close() after running systemd commands.
@dongsupark
dongsupark force-pushed the dongsu/dbus-conn-close-fxtests branch from a0e0ef6 to 951c99f Compare November 15, 2016 15:19
@dongsupark

Copy link
Copy Markdown
Author

As it's clearly a bug, and it doesn't affect main code, I'll simply merge.
BTW fleetd doesn't explicitly close DBUS connections either. Though the remaining connections are already cleaned up when fleetd shuts down. That's what I see in my tests. So I suppose, we don't necessarily need to clean them up in fleetd.

@dongsupark
dongsupark merged commit e6c838b into coreos:master Nov 16, 2016
@dongsupark
dongsupark deleted the dongsu/dbus-conn-close-fxtests branch November 16, 2016 11:16
@jonboulle

Copy link
Copy Markdown
Contributor

nice catch!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants