[PATCH v2 16/23] conf: Move "all" handling to port specifier