[00:04:06] 06cloud-services-team, 10Toolforge: [builds-api,builds-service,builds-cli] toolforge build --envvar does not accept values containing equals character - https://phabricator.wikimedia.org/T389694#10736289 (10Raymond_Ndibe) [00:06:30] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api] Split the `*Job` API models into three - https://phabricator.wikimedia.org/T390136#10736290 (10Raymond_Ndibe) [00:09:02] 06cloud-services-team, 10Toolforge, 13Patch-For-Review: Running out of "secrets" quota (envvars) produces unhelpful error message from `toolforge envvars create` - https://phabricator.wikimedia.org/T364878#10736291 (10Raymond_Ndibe) [00:12:16] 06cloud-services-team, 10Tool-toolviews, 10Toolforge: Provide basic page view metrics for individual tools on toolforge - https://phabricator.wikimedia.org/T87001#10736293 (10Raymond_Ndibe) [00:14:16] 06cloud-services-team, 10Toolforge: [infra,builds-api,harbor] Tool (k8s-status or a new one) to display details about buildservice pipelines and Harbor images - https://phabricator.wikimedia.org/T336133#10736295 (10Raymond_Ndibe) [00:19:33] 06cloud-services-team, 10Toolforge: [jobs-api,jobs-cli] Replace already completed one-off jobs when starting a new one - https://phabricator.wikimedia.org/T352989#10736296 (10Raymond_Ndibe) [00:20:19] 06cloud-services-team, 10Toolforge: [jobs-cli] If the pod exists and it has no logs, read the message status from it and output that - https://phabricator.wikimedia.org/T384251#10736298 (10Raymond_Ndibe) [00:21:05] 06cloud-services-team, 10Toolforge: toolforge webservice logs -f not robust to invalid output - https://phabricator.wikimedia.org/T383742#10736299 (10Raymond_Ndibe) [00:21:50] 06cloud-services-team, 10Toolforge: [functional-tests,deploy,cookbook] Run only selected tests when deploying a component - https://phabricator.wikimedia.org/T381011#10736300 (10Raymond_Ndibe) [00:25:29] 06cloud-services-team, 10Toolforge: [envvars-api, envvars-cli] Create envvar name error message is not user friendly - https://phabricator.wikimedia.org/T360147#10736301 (10Raymond_Ndibe) [00:27:04] 06cloud-services-team, 10Toolforge: [jobs-api,jobs-cli] Expose hidden quota errors more clearly - https://phabricator.wikimedia.org/T333976#10736303 (10Raymond_Ndibe) [00:27:42] 06cloud-services-team, 10Toolforge, 07Kubernetes: [jobs-api,jobs-cli,infra] Transient cronjob scheduling failures on Toolforge k8s - https://phabricator.wikimedia.org/T338006#10736305 (10Raymond_Ndibe) [00:28:03] 06cloud-services-team, 10Toolforge: [jobs-api,jobs-cli] Show a job status when a job is being deleted - https://phabricator.wikimedia.org/T348242#10736306 (10Raymond_Ndibe) [00:31:01] 06cloud-services-team, 10Toolforge: Environment variables are not being passed - https://phabricator.wikimedia.org/T390845#10736308 (10Raymond_Ndibe) [00:33:01] 06cloud-services-team, 10Toolforge: [jobs-emailer] If the pod is in error status, try to get the status.message field in the email, otherwise just 'error' is not that useful - https://phabricator.wikimedia.org/T384252#10736309 (10Raymond_Ndibe) [00:33:46] 06cloud-services-team, 10Toolforge: [lima-kilo] when using "--ha", some containers are not restarting after restarting the VM - https://phabricator.wikimedia.org/T385082#10736310 (10Raymond_Ndibe) [00:35:05] 06cloud-services-team, 10Toolforge: [jobs-api] Indicate when a job is too big to be scheduled - https://phabricator.wikimedia.org/T383515#10736311 (10Raymond_Ndibe) [00:35:51] 06cloud-services-team, 10Toolforge: toolforge jobs load errors with 404 repetatively - https://phabricator.wikimedia.org/T381273#10736312 (10Raymond_Ndibe) [00:38:31] 06cloud-services-team, 10Toolforge: toolforge jobs load errors with 404 repetatively - https://phabricator.wikimedia.org/T381273#10736314 (10Raymond_Ndibe) @Urbanecm is this still an issue? since the creation of this tasks the `toolforge jobs load` gone through at least 2 iterations. Please let us know if this... [00:44:12] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api] Split the `*Job` API models into three - https://phabricator.wikimedia.org/T390136#10736316 (10Raymond_Ndibe) [00:44:18] 10Toolforge (Toolforge iteration 19), 13Patch-For-Review: [jobs-api] refactor models - https://phabricator.wikimedia.org/T389118#10736317 (10Raymond_Ndibe) [00:46:10] (03update) 10raymond-ndibe: [jobs-api] split job models to oneoff, scheduled and continuous [repos/cloud/toolforge/jobs-api] (use_pydantic_for_core_job_model) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/154 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [00:47:05] (03update) 10raymond-ndibe: [jobs-cli] refactor job payload [repos/cloud/toolforge/jobs-cli] (health_check_and_quota_refactor) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/98 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [00:47:24] (03update) 10raymond-ndibe: [jobs-api] split job models to oneoff, scheduled and continuous [repos/cloud/toolforge/jobs-api] (use_pydantic_for_core_job_model) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/154 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [00:47:50] (03update) 10raymond-ndibe: [jobs-cli] refactor job payload [repos/cloud/toolforge/jobs-cli] (health_check_and_quota_refactor) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/98 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [00:49:04] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api] Split the `*Job` API models into three - https://phabricator.wikimedia.org/T390136#10736321 (10Raymond_Ndibe) 05Open→03In progress [00:49:35] 06cloud-services-team, 10Toolforge (Toolforge iteration 19), 13Patch-For-Review: [jobs-api] Split the `*Job` API models into three - https://phabricator.wikimedia.org/T390136#10736324 (10Raymond_Ndibe) [00:49:41] 06cloud-services-team, 10Toolforge (Toolforge iteration 19), 13Patch-For-Review: [harbor, builds-builder] Audit robot account permissions - https://phabricator.wikimedia.org/T361708#10736325 (10Raymond_Ndibe) [01:40:11] (03update) 10raymond-ndibe: [jobs-cli] refactor job payload [repos/cloud/toolforge/jobs-cli] (health_check_and_quota_refactor) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/98 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [01:43:42] (03update) 10raymond-ndibe: [jobs-cli] refactor job payload [repos/cloud/toolforge/jobs-cli] (health_check_and_quota_refactor) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/98 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [02:04:40] 06cloud-services-team, 10Toolforge: [builds-api,builds-service,builds-cli] toolforge build --envvar does not accept values containing equals character - https://phabricator.wikimedia.org/T389694#10736350 (10Raymond_Ndibe) I looked on the tekton side and I got this: ` - args: - -wait_file - /tekton/ru... [02:23:37] 06cloud-services-team, 10Toolforge: [jobs-cli] If the pod exists and it has no logs, read the message status from it and output that - https://phabricator.wikimedia.org/T384251#10736358 (10Raymond_Ndibe) @dcaro can we talk about the use case of this a bit more? Isn't what we are describing here what `status_lo... [02:27:57] 06cloud-services-team, 10Toolforge: [jobs-api,jobs-cli] Show a job status when a job is being deleted - https://phabricator.wikimedia.org/T348242#10736360 (10Raymond_Ndibe) I think we can close this now. `toolforge jobs flush/delete` now waits until the job(s) is gone. [04:07:36] 06cloud-services-team, 10Toolforge: [jobs-cli] If the pod exists and it has no logs, read the message status from it and output that - https://phabricator.wikimedia.org/T384251#10736387 (10Raymond_Ndibe) btw this is related to this task https://phabricator.wikimedia.org/T349775 no? I feel like fixing the linke... [04:20:51] 06cloud-services-team, 10Toolforge: [functional-tests,deploy,cookbook] Run only selected tests when deploying a component - https://phabricator.wikimedia.org/T381011#10736390 (10Raymond_Ndibe) Is the priority here really `medium`? I know that we can use `--filter-tag` to only run select tests (this task is rea... [04:33:39] 06cloud-services-team, 10Toolforge, 13Patch-For-Review: Running out of "secrets" quota (envvars) produces unhelpful error message from `toolforge envvars create` - https://phabricator.wikimedia.org/T364878#10736394 (10Raymond_Ndibe) closing this as this is no longer an issue. I ran a script to create multip... [04:34:00] 06cloud-services-team, 10Toolforge, 13Patch-For-Review: Running out of "secrets" quota (envvars) produces unhelpful error message from `toolforge envvars create` - https://phabricator.wikimedia.org/T364878#10736395 (10Raymond_Ndibe) 05Open→03Resolved [04:56:59] (03open) 10raymond-ndibe: [envvars-cli] hide envvar if truncate true [repos/cloud/toolforge/envvars-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-cli/-/merge_requests/78 (https://phabricator.wikimedia.org/T363544) [04:57:05] (03update) 10raymond-ndibe: [envvars-cli] hide envvar if truncate true [repos/cloud/toolforge/envvars-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-cli/-/merge_requests/78 (https://phabricator.wikimedia.org/T363544) [05:07:03] 06cloud-services-team, 10Toolforge (Toolforge iteration 19), 13Patch-For-Review: [envvars-cli] Add option to not show envvar values when listing - https://phabricator.wikimedia.org/T363544#10736398 (10Raymond_Ndibe) [05:07:11] 06cloud-services-team, 10Toolforge (Toolforge iteration 19), 13Patch-For-Review: [envvars-cli] Add option to not show envvar values when listing - https://phabricator.wikimedia.org/T363544#10736400 (10Raymond_Ndibe) 05Open→03In progress [05:07:24] 06cloud-services-team, 10Toolforge (Toolforge iteration 19), 13Patch-For-Review: [envvars-cli] Add option to not show envvar values when listing - https://phabricator.wikimedia.org/T363544#10736401 (10Raymond_Ndibe) a:03Raymond_Ndibe [08:28:02] (03open) 10raymond-ndibe: [envvars-api] return custom message for invalid EnvvarName [repos/cloud/toolforge/envvars-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-api/-/merge_requests/53 [08:41:48] (03update) 10raymond-ndibe: [envvars-api] return custom message for invalid EnvvarName [repos/cloud/toolforge/envvars-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-api/-/merge_requests/53 [08:45:09] (03update) 10raymond-ndibe: [envvars-api] return custom message for invalid EnvvarName [repos/cloud/toolforge/envvars-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-api/-/merge_requests/53 [09:24:10] (03update) 10raymond-ndibe: [envvars-api] return custom message for invalid EnvvarName [repos/cloud/toolforge/envvars-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-api/-/merge_requests/53 [09:39:59] (03update) 10raymond-ndibe: [envvars-api] return custom message for invalid EnvvarName [repos/cloud/toolforge/envvars-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-api/-/merge_requests/53 [12:42:56] 06cloud-services-team, 10Toolforge: [builds-api,builds-service,builds-cli] toolforge build --envvar does not accept values containing equals character - https://phabricator.wikimedia.org/T389694#10736690 (10Don-vip) Hi @Raymond_Ndibe, To reproduce you can `become spacemedia` and trigger a new build using `./... [13:15:41] (03open) 10don-vip: Draft: T389694 - rework envvar parsing to keep text after an additional '=' character [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/70 [13:16:40] 06cloud-services-team, 10Toolforge: [builds-api,builds-service,builds-cli] toolforge build --envvar does not accept values containing equals character - https://phabricator.wikimedia.org/T389694#10736704 (10Don-vip) @Raymond_Ndibe I think I have found the problem, can you please take a look at https://gitlab.w... [13:20:13] (03update) 10don-vip: T389694 - rework envvar parsing to keep text after an additional '=' character [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/70 [14:34:18] 10Tools, 10Wikidata, 07Security: Blocked Wikidata user sockpuppets are doing automated misconduct with QuickStatements - https://phabricator.wikimedia.org/T386978#10736720 (10Bugreporter) Only checking whether the user is blocked only when an edit fails would be better. [18:14:16] 06cloud-services-team, 10Cloud-VPS: network issues with new VMs in codfw1dev - https://phabricator.wikimedia.org/T391727#10736838 (10Andrew) 05Open→03Invalid This was a lack of egress rules in new projects. Probably a bug, but not the bug I thought it was! [18:33:41] FIRING: CloudVPSDesignateLeaks: Detected 2 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [18:42:34] 10Tool-documentation, 10Adiutor, 06Wiki-Mentor-Africa, 07good first task: [Documentation] Update and improve docs for Adiutor tool - https://phabricator.wikimedia.org/T390392#10736858 (10CBID2) @apaskulin, can you help me with this? [18:43:41] RESOLVED: CloudVPSDesignateLeaks: Detected 2 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [22:46:39] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [22:49:08] !log andrew@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment codfw1dev for all services