[06:39:26] https://idp.wikimedia.org/ is down? [06:53:36] moritzm: ^ [06:58:17] I have restarted tomcat, and it seems to be back [07:18:47] marostegui: ack, thx [14:39:20] !incidents [14:39:20] 5560 (ACKED) [2x] ProbeDown sre (wikikube-ctrl1004:6443 probes/custom eqiad) [14:39:21] 5561 (RESOLVED) ATSBackendErrorsHigh cache_upload sre (swift.discovery.wmnet eqsin) [14:39:21] 5559 (RESOLVED) ATSBackendErrorsHigh cache_upload sre (swift.discovery.wmnet eqsin) [14:39:21] 5558 (RESOLVED) ATSBackendErrorsHigh cache_upload sre (swift.discovery.wmnet eqsin) [14:39:21] 5557 (RESOLVED) ATSBackendErrorsHigh cache_upload sre (swift.discovery.wmnet eqsin) [15:14:09] has anyone gotten podman rootless to work on WMCS (or with LDAP/SSSD in general)? I'm getting `potentially insufficient UIDs or GIDs available in user namespace` even after I add entries to `/etc/subuid` and `/etc/subgid` I think https://github.com/containers/podman/discussions/16244#discussioncomment-5697346 might be relevant but LMK [15:16:38] inflatador: do check that user namespaces aren't forbidden [15:16:50] we have a puppet fleet wide setting to disable that [15:17:02] I don't know whether WMCS Cloud VPS inherits it [15:17:52] Yeah, seems they do, but I did set `kernel.unprivileged_userns_clone = 1` [15:18:06] oh, if you already did, disregard [15:18:21] I think there is one more setting for max user namespaces...let me check [15:18:45] user.max_user_namespaces = 62263 [15:18:52] which we don't touch AFAICT [15:19:01] and should default to that ^ [15:19:33] yeah, confirmed [15:19:58] `user.max_user_namespaces = 16384` [15:33:06] I swear this used to work, but maybe I'm hallucinating