[00:02:27] 10Data-Engineering, 10DC-Ops, 10Infrastructure-Foundations: clouddb1021 missing firmware: debian installer cannot connect to network - https://phabricator.wikimedia.org/T306148 (10razzi) Possibly relevant links thanks to @jhathaway: https://packages.debian.org/bullseye/firmware-bnx2x does not contain the re... [03:31:01] PROBLEM - Check unit status of eventlogging_to_druid_editattemptstep_hourly on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit eventlogging_to_druid_editattemptstep_hourly https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [04:04:45] RECOVERY - Check unit status of eventlogging_to_druid_editattemptstep_hourly on an-launcher1002 is OK: OK: Status of the systemd unit eventlogging_to_druid_editattemptstep_hourly https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [06:50:16] 10Data-Engineering, 10DC-Ops, 10Infrastructure-Foundations: clouddb1021 missing firmware: debian installer cannot connect to network - https://phabricator.wikimedia.org/T306148 (10razzi) @marostegui says I should tag @MoritzMuehlenhoff - hopefully we can all solve this together :) [06:50:25] 10Data-Engineering, 10DC-Ops, 10Infrastructure-Foundations: clouddb1021 missing firmware; debian installer cannot connect to network - https://phabricator.wikimedia.org/T306148 (10razzi) [07:10:29] PROBLEM - Hadoop NodeManager on an-worker1093 is CRITICAL: PROCS CRITICAL: 0 processes with command name java, args org.apache.hadoop.yarn.server.nodemanager.NodeManager https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts%23Yarn_Nodemanager_process [07:35:03] RECOVERY - Hadoop NodeManager on an-worker1093 is OK: PROCS OK: 1 process with command name java, args org.apache.hadoop.yarn.server.nodemanager.NodeManager https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts%23Yarn_Nodemanager_process [08:08:11] 10Data-Engineering, 10DC-Ops, 10Infrastructure-Foundations: clouddb1021 missing firmware; debian installer cannot connect to network - https://phabricator.wikimedia.org/T306148 (10ayounsi) FYI this host is alerting in Netbox: > clouddb1021 Device is Active in Netbox but is missing from PuppetDB (should be... [08:14:13] 10Data-Engineering, 10Data-Engineering-Kanban, 10Beta-Cluster-Infrastructure, 10Event-Platform: Upgrade event platform related VMs in deployment-prep to Debian bullsye (or buster) - https://phabricator.wikimedia.org/T304433 (10Jdforrester-WMF) [08:41:56] (03PS10) 10Aqu: Fix: Prevent empty normalized host [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) [09:46:45] (03PS11) 10Aqu: Fix: Prevent empty normalized host [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) [09:54:56] 10Data-Engineering, 10Data-Catalog, 10SRE, 10serviceops, and 2 others: New Service Request: DataHub - https://phabricator.wikimedia.org/T303049 (10BTullis) datahub.wikimedia.org is now up {F35051150,width=60%} Now working on getting the datahub-gms.discovery.wmnet service up and running too. [11:07:26] (03PS12) 10Aqu: Fix: Prevent empty normalized host [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) [11:23:43] 10Data-Engineering, 10Airflow: Low RIsk Ozzie Migration: Wikidata CoEditor metric job - https://phabricator.wikimedia.org/T306177 (10Snwachukwu) [11:25:12] 10Data-Engineering, 10Airflow: Low RIsk Ozzie Migration: Wikidata CoEditor metric job - https://phabricator.wikimedia.org/T306177 (10Snwachukwu) [11:38:58] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog, 10Patch-For-Review: Configure LDAP authentication for the DataHub frontend - https://phabricator.wikimedia.org/T301462 (10BTullis) This is now confirmed working in production. {F35051305,width=60%} [11:40:23] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Configure LDAP authentication for the DataHub frontend - https://phabricator.wikimedia.org/T301462 (10BTullis) [12:16:44] 10Analytics, 10Traffic: intake-analytics is responsible for up to a 85% of varnish backend fetch errors - https://phabricator.wikimedia.org/T306181 (10Vgutierrez) [12:25:44] 10Data-Engineering, 10Generated Data Platform, 10SRE, 10SRE-Access-Requests: Request to grant cparle and mfossati login to an-airflow1003.eqiad.wmne - https://phabricator.wikimedia.org/T306057 (10Ottomata) https://www.mediawiki.org/wiki/Readers/Structured_Data [12:30:58] (03PS1) 10Snwachukwu: Create Hive Query to generate Wikidata CoEditors metrics [analytics/refinery] - 10https://gerrit.wikimedia.org/r/780749 (https://phabricator.wikimedia.org/T306177) [12:32:44] (03CR) 10Snwachukwu: "Kindly help review the Hive Query to generate Wikidata CoEditor metric" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/780749 (https://phabricator.wikimedia.org/T306177) (owner: 10Snwachukwu) [12:53:38] (03PS13) 10Aqu: Make caching mechanisms thread ready [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) [12:58:02] (03CR) 10Aqu: [V: 03+1 C: 03+1] "Thanks Gehel, Joal, Ottomata for your comments." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) (owner: 10Aqu) [13:15:13] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Documentation, 10cloud-services-team (Kanban): Document on wikitech the general process of getting a table/column exposed to Wiki Replica users - https://phabricator.wikimedia.org/T209992 (10JArguello-WMF) 05Open→03Declined No longer n... [13:38:59] (03CR) 10Milimetric: "just some thoughts and questions, I think the job would be fine to merge as-is, it's a faithful reproduction of the original." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/780749 (https://phabricator.wikimedia.org/T306177) (owner: 10Snwachukwu) [13:46:34] 10Analytics, 10SRE, 10Traffic: intake-analytics is responsible for up to a 85% of varnish backend fetch errors - https://phabricator.wikimedia.org/T306181 (10Vgutierrez) p:05Triage→03Medium [14:17:01] 10Data-Engineering, 10Data-Engineering-Kanban, 10Airflow, 10Patch-For-Review: Create Generic Hive-To-Graphite job - https://phabricator.wikimedia.org/T304623 (10Snwachukwu) [14:19:21] 10Data-Engineering, 10Data-Engineering-Kanban, 10Airflow: Create Generic Hive-To-Graphite job - https://phabricator.wikimedia.org/T304623 (10Snwachukwu) [14:27:41] (03PS14) 10Aqu: Make caching mechanisms thread ready [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) [14:42:23] PROBLEM - Check unit status of produce_canary_events on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit produce_canary_events https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [14:52:59] ottomata: yt? o/ [14:53:37] RECOVERY - Check unit status of produce_canary_events on an-launcher1002 is OK: OK: Status of the systemd unit produce_canary_events https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [15:15:03] btullis: hello! [15:16:05] Hiya. I was wondering, have you ever had to deal with validating wmf-certificates in conda environments? If so, any tips? [15:18:31] I have a situation where I'm using `pip install 'acryl-datahub[datahub]'` in a conda environment, but because https://datahub-gms.discovery.wmnet is ultimately signed by our own WMF Root CA, it fails validation. [15:19:35] (03CR) 10David Caro: [C: 03+1] "LGTM, just nits" [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/773578 (https://phabricator.wikimedia.org/T100982) (owner: 10Vivian Rook) [15:20:05] hmmm [15:20:45] btullis: no but what if you do [15:20:48] apt-get install wmf-certificates [15:21:27] https://gerrit.wikimedia.org/r/plugins/gitiles/operations/debs/wmf-certificates/+/refs/heads/main/debian/control [15:22:12] btullis: why is does pip install need the wmf root ca though? [15:22:16] why does* [15:22:34] or do you mean that you can't use acryl-datahub[datahub] in the condaenv after it is installed [15:22:36] ? [15:22:50] The latter. It's not pip install that needs it, it's `datahub ingest list-runs` (and similar). [15:24:34] aye [15:25:14] btullis: are you trying to do that locally or in on a prod node somewhere? [15:25:50] I tried copying `/etc/ssl/certs/ca-certificate.crt` over the top of `~/.conda/$env/ssl/cecert.pem` - it sort of worked for some things, but not quite enough. [15:26:05] I'm working on stat1008 at the moment. [15:26:17] ohhhh [15:26:19] hm [15:26:22] i see [15:26:52] hmmm [15:27:31] I just wondered whether you'd seen it before, because it caught me out :-) [15:27:32] btullis: is this an anaconda stacked [15:27:35] or a standalone one? [15:27:40] env* [15:28:35] I've tried anaconda (base) and a stacked one. I've even tried a `python -m venv ./datahub`as well. [15:28:54] okay [15:28:54] yes [15:29:26] i think we need to add something to REQUESTS_CA_BUNDLE that is set when you activate a stacked env [15:29:39] can you try [15:31:36] REQUESTS_CA_BUNDLE=$REQUESTS_CA_BUNDLE:/usr/share/ca-certificates/wikimedia/Wikimedia_Internal_Root_CA.crt [15:34:22] actually, i think a better value is [15:34:26] /etc/ssl/certs/wmf-ca-certificates.crt [15:34:37] REQUESTS_CA_BUNDLE=$REQUESTS_CA_BUNDLE: /etc/ssl/certs/wmf-ca-certificates.crt [15:35:36] REQUESTS_CA_BUNDLE=$REQUESTS_CA_BUNDLE:/etc/ssl/certs/wmf-ca-certificates.crt [15:35:37] hmm wait [15:35:41] does it take multiple values? [15:35:42] not suer. [15:37:35] hmmm no. [15:37:36] i'm wrong [15:37:49] the REQUESTS_CA_BUNDLE value that should be set when you activate your conda env should be correct [15:38:09] it should be /etc/ssl/certs/ca-certificates.crt [15:38:12] which should have the wmf certs in it [15:38:21] btullis: what do you get for [15:38:24] echo $REQUESTS_CA_BUNDLE [15:38:25] ? [15:38:49] null [15:39:23] This is after activating a freshly minted conda env with: `source /home/btullis/.conda/envs/2022-04-14T15.32.43_btullis/bin/activate` [15:41:54] I tried this, but it was the same. Unable to get local issuer certificate. [15:41:57] https://www.irccloud.com/pastebin/HDCupzAU/ [15:45:24] (03PS1) 10Btullis: Add a new config file to the datahub-gms image [analytics/datahub] (wmf) - 10https://gerrit.wikimedia.org/r/780884 (https://phabricator.wikimedia.org/T306019) [15:53:05] 10Data-Engineering: Cleanup analytics/refinery/source pom.files - https://phabricator.wikimedia.org/T306193 (10Antoine_Quhen) [15:55:04] 10Quarry, 10Epic: [Epic] Make Quarry a robust data exploration tool - https://phabricator.wikimedia.org/T194330 (10rook) Closing for now. If anyone wants to revive this please feel free to reopen. [15:56:03] 10Quarry, 10Epic: [Epic] Make Quarry a robust data exploration tool - https://phabricator.wikimedia.org/T194330 (10rook) 05Open→03Declined [15:56:10] (03PS1) 10Aqu: Cleanup dependencies of core/pom.xml file [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/780886 (https://phabricator.wikimedia.org/T306193) [15:59:05] 10Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Patch-For-Review, 10Readers-Web-Backlog (Needs Prioritization (Tech)): Deprecate/delete the mw.eventLog.Schema class - https://phabricator.wikimedia.org/T305491 (10phuedx) [15:59:12] 10Quarry: Add current git version somewhere in the interface - https://phabricator.wikimedia.org/T192506 (10rook) [15:59:18] 10Quarry, 10Cloud-Services-Origin-Team, 10Cloud-Services-Worktype-Project, 10User-dcaro, 10cloud-services-team (Kanban): [quarry] Add 'feedback' link to pre-filled phabricator task - https://phabricator.wikimedia.org/T303028 (10rook) [15:59:25] (03CR) 10Aqu: Make caching mechanisms thread ready (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) (owner: 10Aqu) [15:59:35] 10Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Patch-For-Review, 10Readers-Web-Backlog (Needs Prioritization (Tech)): Deprecate/delete the mw.eventLog.Schema class - https://phabricator.wikimedia.org/T305491 (10phuedx) > If we decide to deprecate the class, announce the deprecation on wikitech-... [16:00:50] 10Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Patch-For-Review, 10Readers-Web-Backlog (Needs Prioritization (Tech)): Deprecate/delete the mw.eventLog.Schema class - https://phabricator.wikimedia.org/T305491 (10Jdlrobson) I can review and merge the above patch. I have set a reminder for 2 weeks... [16:02:29] 10Quarry: Provide option to download all user queries - https://phabricator.wikimedia.org/T192209 (10rook) [16:02:43] 10Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Patch-For-Review, 10Readers-Web-Backlog (Needs Prioritization (Tech)): Deprecate/delete the mw.eventLog.Schema class - https://phabricator.wikimedia.org/T305491 (10DLynch) Your codesearch is missing things -- it's used in MobileFrontend in schemaEd... [16:02:59] 10Quarry, 10cloud-services-team (Kanban): Support queries against Quarry's own database and ToolsDB - https://phabricator.wikimedia.org/T151158 (10rook) [16:05:31] btullis: strange that it is null... [16:05:38] cat /home/btullis/.conda/envs/2022-04-14T15.32.43_btullis/etc/conda/activate.d/env_vars.sh [16:09:54] (03CR) 10Btullis: [C: 03+2] Add a new config file to the datahub-gms image [analytics/datahub] (wmf) - 10https://gerrit.wikimedia.org/r/780884 (https://phabricator.wikimedia.org/T306019) (owner: 10Btullis) [16:10:53] 10Data-Engineering-Kanban, 10Airflow, 10Patch-For-Review: Low Risk Oozie Migration: browser/general - https://phabricator.wikimedia.org/T302875 (10Milimetric) [16:12:34] 10Data-Engineering, 10Data-Engineering-Kanban, 10Airflow, 10Patch-For-Review: Low RIsk Ozzie Migration: Wikidata CoEditor metric job - https://phabricator.wikimedia.org/T306177 (10Milimetric) [16:23:31] 10Quarry: Huge resultsets may cause issues - https://phabricator.wikimedia.org/T151114 (10rook) Currently only 100 rows are shown at a time. This may have been updated through another patch. Please re-open if this is still an issue. [16:23:39] 10Quarry: Huge resultsets may cause issues - https://phabricator.wikimedia.org/T151114 (10rook) 05Open→03Resolved [16:26:46] btullis: let's figure that out together [16:33:04] (03Merged) 10jenkins-bot: Add a new config file to the datahub-gms image [analytics/datahub] (wmf) - 10https://gerrit.wikimedia.org/r/780884 (https://phabricator.wikimedia.org/T306019) (owner: 10Btullis) [16:40:34] 10Data-Engineering: Fix anaconda-wmf's setting of REQUESTS_CA_BUNDLE - https://phabricator.wikimedia.org/T306197 (10Ottomata) [16:44:30] (03PS1) 10David Caro: Add links to feedback and bug pre-filled tasks [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/780899 (https://phabricator.wikimedia.org/T303028) [16:46:50] (03PS2) 10David Caro: Add links to feedback and bug pre-filled tasks [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/780899 (https://phabricator.wikimedia.org/T303028) [16:57:49] ottomata: Great work on finding that bug and filing the patch. Thanks :-) [16:59:12] (VarnishkafkaNoMessages) firing: ... [16:59:18] varnishkafka for instance cp2027:9132 is not logging cache_text requests from eventlogging - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-source=eventlogging&var-cp_cluster=cache_text&var-instance=cp2027:9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [17:01:21] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Update DNS for the DataHub MVP services - https://phabricator.wikimedia.org/T301460 (10BTullis) I'm calling this one done, although there is a caveat on [[https://gerrit.wikimedia.org/r/c/operations/dns/+/780658/|this change]] which states that t... [17:02:11] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Configure LDAP authentication for the DataHub frontend - https://phabricator.wikimedia.org/T301462 (10BTullis) 05Open→03Resolved [17:02:13] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog MVP - https://phabricator.wikimedia.org/T299910 (10BTullis) [17:15:14] ty btullis ! [17:15:21] sorry for the stupidness :) [19:24:36] 10Quarry: Too much whitespace in Quarry - https://phabricator.wikimedia.org/T112803 (10rook) This may have been updated since 2015, looks, in my subjective view, alright to me. Please reopen if this needs further addressing. [19:24:45] 10Quarry: Too much whitespace in Quarry - https://phabricator.wikimedia.org/T112803 (10rook) 05Open→03Resolved [19:31:02] 10Quarry: Some long queries give no results - https://phabricator.wikimedia.org/T109016 (10rook) This has perhaps changed, I'm not seeing a log_comment column. Please re-open if what is seen can be reproduced. [19:31:12] 10Quarry: Some long queries give no results - https://phabricator.wikimedia.org/T109016 (10rook) 05Open→03Declined [19:33:04] 10Quarry: Large queries results not displaying (Aug 2015) - https://phabricator.wikimedia.org/T108084 (10rook) Linked query is showing 30913 rows, this ticket may have been forgotten. Please re-open if there is any additional work to be done on this. [19:33:12] 10Quarry: Large queries results not displaying (Aug 2015) - https://phabricator.wikimedia.org/T108084 (10rook) 05Open→03Resolved [19:37:15] 10Quarry: Have an easy way to ban users from Quarry - https://phabricator.wikimedia.org/T104322 (10rook) I agree with @Tgr is this not something to handle through stewards? [19:38:53] 10Quarry, 10Patch-For-Review: Add list of query executions to the query page side-bar - https://phabricator.wikimedia.org/T100982 (10rook) a:03rook [19:48:10] 10Quarry, 10Tool-tsreports: Provide redirects from old tsreports reports to new quarry reports - https://phabricator.wikimedia.org/T78595 (10rook) I don't believe tsreports exists sufficiently to do this. https://tsreports.toolforge.org/ mostly give a link pointing to quarry as an alternative. Please re-open i... [19:48:25] 10Quarry, 10Tool-tsreports: Provide redirects from old tsreports reports to new quarry reports - https://phabricator.wikimedia.org/T78595 (10rook) 05Open→03Declined [19:48:28] 10Quarry, 10Tool-tsreports: Quarry-TSreports feature parity - https://phabricator.wikimedia.org/T78549 (10rook) [19:50:37] 10Quarry: Ideas for reports - https://phabricator.wikimedia.org/T78597 (10rook) [19:50:41] 10Quarry, 10Tool-tsreports: Quarry-TSreports feature parity - https://phabricator.wikimedia.org/T78549 (10rook) [19:58:02] 10Quarry: Reports: Add a thumbnail field - https://phabricator.wikimedia.org/T60844 (10rook) [19:58:13] 10Quarry: Provide a way to add hyperlink in Quarry results/output - https://phabricator.wikimedia.org/T74874 (10rook) [19:58:42] 10Quarry: Reports: Add a thumbnail field - https://phabricator.wikimedia.org/T60844 (10rook) This seems a near duplicate of T74874, if we need the image itself please reopen this ticket. [20:01:38] 10Quarry, 10Patch-For-Review: Add database selector - https://phabricator.wikimedia.org/T76466 (10rook) Quarry now has a database selector field. I lack tsreports knowledge to know if it is the intent. If more than quarry currently offers is needed please reopen this ticket or a separate ticket with additional... [20:01:41] 10Quarry, 10Patch-For-Review: Add database selector - https://phabricator.wikimedia.org/T76466 (10rook) 05Open→03Resolved [20:01:45] 10Quarry: Build a schema explorer - https://phabricator.wikimedia.org/T71192 (10rook) [20:02:46] 10Quarry: Categorizing the queries - https://phabricator.wikimedia.org/T76127 (10rook) [20:02:54] 10Quarry: Search or filter queries by title or summary - https://phabricator.wikimedia.org/T90509 (10rook) [20:10:14] 10Quarry: Make the table sortable - https://phabricator.wikimedia.org/T71265 (10rook) [20:10:25] 10Quarry: Search or filter queries by title or summary - https://phabricator.wikimedia.org/T90509 (10rook) [20:59:27] (VarnishkafkaNoMessages) firing: ... [20:59:27] varnishkafka for instance cp2027:9132 is not logging cache_text requests from eventlogging - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-source=eventlogging&var-cp_cluster=cache_text&var-instance=cp2027:9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [21:27:00] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Connect MVP to Hive metastore [Mile Stone 4] - https://phabricator.wikimedia.org/T299897 (10Milimetric) Ben got everything running! I'm starting Hive Ingestion. First shot, just `event_sanitized` with profiling on to see how bad it is ` source... [21:34:19] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Connect MVP to Hive metastore [Mile Stone 4] - https://phabricator.wikimedia.org/T299897 (10Milimetric) Errors like this: `[2022-04-14 21:26:11,826] ERROR {datahub.ingestion.run.pipeline:93} - failed to write record with workunit event_saniti... [22:01:10] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Connect MVP to Hive metastore [Mile Stone 4] - https://phabricator.wikimedia.org/T299897 (10BTullis) Thanks Dan. I wonder if it's anything to do with karapace, as opposed to schema-registry. We can look at all of the application logs in Logstash... [22:03:03] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Connect MVP to Hive metastore [Mile Stone 4] - https://phabricator.wikimedia.org/T299897 (10BTullis) You could also try another test with an output sink of 'console' instead of 'datahub-rest'. Eyeball it at this point. [22:44:28] (03CR) 10Gergő Tisza: Image Suggestions feature schema (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/779052 (owner: 10Luke Bowmaker) [22:45:04] (03PS5) 10Gergő Tisza: Image Suggestions feature schema [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/779052 (owner: 10Luke Bowmaker)