[00:31:37] RECOVERY - Check systemd state on an-launcher1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [00:31:39] RECOVERY - Check unit status of monitor_refine_eventlogging_legacy on an-launcher1002 is OK: OK: Status of the systemd unit monitor_refine_eventlogging_legacy https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [06:53:43] (03CR) 10Joal: [C: 03+2] "Merging for deploy this week" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/807340 (https://phabricator.wikimedia.org/T138093) (owner: 10Ori) [07:01:59] (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for future deploy" [analytics/wikihadoop] - 10https://gerrit.wikimedia.org/r/624745 (owner: 10Joal) [07:04:17] (03Merged) 10jenkins-bot: UDF for testing uri_query for duplicate query parameters [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/807340 (https://phabricator.wikimedia.org/T138093) (owner: 10Ori) [07:04:55] (03PS1) 10Joal: Bump hadoop dependencies version to 2.10.1 [analytics/wikihadoop] - 10https://gerrit.wikimedia.org/r/808791 [07:18:12] (03PS1) 10Joal: Bump scala version to 2.12.10 [analytics/wikihadoop] - 10https://gerrit.wikimedia.org/r/808793 [07:18:14] (03PS1) 10Joal: Bump scala version to 2.12.10 and fix pom [analytics/wikihadoop] - 10https://gerrit.wikimedia.org/r/808794 [07:26:07] 10Data-Engineering, 10Data-Persistence (Consultation): Move Mediawiki QueryPages computation to Hadoop - https://phabricator.wikimedia.org/T309738 (10Ladsgroup) Here is list of classes using QueryPage: https://doc.wikimedia.org/mediawiki-core/master/php/classQueryPage.html not all are slow and lots of them ge... [08:05:24] I would love to chat about CentralNotice banner impression analytics, for WMDE in the context of T298777. My two questions are, * whether there's already an analytics cluster data stream set up for this, and * if not, then what's the recommended way to get hourly dumps of a subset of webrequest? [08:05:25] T298777: Prevent gaps when providing banner impression data of WMDE fundraising banners - https://phabricator.wikimedia.org/T298777 [09:17:00] awight: I can try to help, but I think that other people are probably far more knowledgable than I am in this area, so I'm not too confident in any of my answers. [09:22:46] I think that there is only a very low sampling rate (1%) on centralnoticeimpressions when using the event based system, so your technique of extracting data from `wmf.webrequest`(as mentioned in T283015) seems like the most detailed method available at the moment. [09:22:47] T283015: Provide aggregated banner impression counts - https://phabricator.wikimedia.org/T283015 [09:35:11] awight: I can see that this topic has been around for a long time without a really comprehensive answer. e.g. T243092 - I'm afraid I don't feel qualified to give a more comprehensive answer about the best way to proceed. Sorry. [09:35:11] T243092: Find a better way for WMDE to get impression counts for their banners - https://phabricator.wikimedia.org/T243092 [10:01:27] btullis: Good point about the sampling, thanks. [11:52:04] btullis: I see that refinery/oozie/banner_activity is already extracting webrequests into a Druid table--but can you clarify whether this data is only updated once per day, or hourly? [12:08:12] 10Data-Engineering: Generate data to count langswitches for every article - https://phabricator.wikimedia.org/T310975 (10JArguello-WMF) [12:08:42] 10Data-Engineering: Add Ukrainian Wikipedia to Clickstream dataset - https://phabricator.wikimedia.org/T310972 (10JArguello-WMF) [12:10:41] 10Data-Engineering-Radar, 10Event-Platform, 10Internet-Archive, 10The-Wikipedia-Library: Page-links-change stream doesn't capture duplicated links - https://phabricator.wikimedia.org/T216492 (10JArguello-WMF) [12:11:17] 10Data-Engineering, 10Pageviews-API: Endpoint for average view rate in Pageview API - https://phabricator.wikimedia.org/T162933 (10JArguello-WMF) [12:11:32] 10Data-Engineering, 10GraphQL, 10Pageviews-API: Responses on pageview API should be lighter - https://phabricator.wikimedia.org/T145935 (10JArguello-WMF) [12:13:09] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q4:(Need By: TBD) rack/setup/install an-presto10[06-15].eqiad.wmnet - https://phabricator.wikimedia.org/T306835 (10BTullis) @Cmjohnson - I think we should just use bullseye for these hosts, if there's a controller issue with buster. We should be able to get... [12:13:19] 10Analytics: Reportupdater writes a README in the output folder - https://phabricator.wikimedia.org/T163134 (10JArguello-WMF) 05Open→03Declined [12:13:21] 10Analytics: reportupdater TLC - https://phabricator.wikimedia.org/T193167 (10JArguello-WMF) [12:13:46] 10Analytics: Improve joining mechanism between webrequest data and edit data for i.e. sampling pageviews - https://phabricator.wikimedia.org/T126290 (10JArguello-WMF) 05Open→03Declined [12:13:55] 10Analytics: Pivot "MediaWiki history" data lake: Feature request for "Event Users" - https://phabricator.wikimedia.org/T161185 (10JArguello-WMF) 05Open→03Declined [12:14:12] 10Analytics: Adding breakdowns to mw edit history reconstruction: wiki projects, categories (cohort) - https://phabricator.wikimedia.org/T163113 (10JArguello-WMF) 05Open→03Declined [12:14:23] 10Data-Engineering, 10Event-Platform, 10EventStreams: Implement server side filtering for EventStreams (if we should) - https://phabricator.wikimedia.org/T152731 (10JArguello-WMF) [12:14:32] 10Analytics: Create table in hive with a continent lookup for countries - https://phabricator.wikimedia.org/T127995 (10JArguello-WMF) 05Open→03Declined [12:14:39] 10Analytics: Update passport-mediawiki module URLs and documentation - https://phabricator.wikimedia.org/T113234 (10JArguello-WMF) 05Open→03Declined [12:14:54] 10Analytics: Dashboard Directory Design Feedback - https://phabricator.wikimedia.org/T92502 (10JArguello-WMF) 05Open→03Declined [12:15:06] 10Analytics: Find out what happens to the old rows in the revision table - https://phabricator.wikimedia.org/T142535 (10JArguello-WMF) 05Open→03Declined [12:16:34] 10Analytics: Pivot - Article Page Views - https://phabricator.wikimedia.org/T166331 (10JArguello-WMF) 05Open→03Declined [12:17:31] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Remove links in wikistats to minnan.wikipedia.org - https://phabricator.wikimedia.org/T107250 (10JArguello-WMF) 05Open→03Declined [12:17:32] awight: OK, thanks for that. I believe that the data is loaded daily, but it has activity at minute detail as per https://github.com/wikimedia/analytics-refinery/blob/master/oozie/banner_activity/druid/README.md [12:19:26] btullis: Do you know if I can tweak this job to provide hourly updates? [12:21:48] It seems feasible to me, but I would have to defer to m'colleagues (joal, mforns, otto etc) as to the implications. [12:23:01] Thanks--I'll also ask the WMDE people (kai) whether having the druid table will satisfy their requirements. [12:23:08] 10Quarry: Quarry database name - https://phabricator.wikimedia.org/T311350 (10rook) While it does seem likely that enwiki_p is the most used, I don't know that we should preference it over other DBs. I also suspect you're right that for a lot of people they use pretty much one DB when they use quarry, so maybe a... [12:24:30] 10Quarry: update quarry page - https://phabricator.wikimedia.org/T311404 (10rook) [12:24:54] 10Quarry: Quarry database name - https://phabricator.wikimedia.org/T311350 (10rook) https://phabricator.wikimedia.org/T311404 Progress! [12:32:58] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad, 10Patch-For-Review: Q4: rack/setup/install stat1010 - https://phabricator.wikimedia.org/T307399 (10BTullis) Thanks @Cmjohnson for your replies. I've created a new partman recipe in https://gerrit.wikimedia.org/r/808870 although I haven't yet tested it. If... [12:43:00] 10Quarry: Quarry database name - https://phabricator.wikimedia.org/T311350 (10LucasWerkmeister) > Every time I create a Quarry query, I need to type enwiki_p. This is a minor irritation for me Ich finde den Vorschlag, die englischsprachige Wikipedia zur Standard-Wikipedia in Quarry zu erheben, deutlich irritier... [12:45:09] 10Quarry: Quarry database name - https://phabricator.wikimedia.org/T311350 (10rook) @LucasWerkmeister +1 [12:57:02] (03PS1) 10NOkafor: Added repair_partitions.hql file to hql path [analytics/refinery] - 10https://gerrit.wikimedia.org/r/808888 (https://phabricator.wikimedia.org/T309718) [13:18:15] (03PS2) 10NOkafor: Updated the usage from hive to spark2-sql Added repair_partitions.hql file to hql path [analytics/refinery] - 10https://gerrit.wikimedia.org/r/808888 (https://phabricator.wikimedia.org/T309718) [13:28:04] 10Quarry, 10Documentation: update quarry page - https://phabricator.wikimedia.org/T311404 (10Aklapper) @rook: I'd love to merge this into T307011 where I already have a question. Okay with that? :) +#documentation [13:29:29] 10Quarry, 10Documentation: update quarry page - https://phabricator.wikimedia.org/T311404 (10rook) Sounds perfect! [13:29:58] 10Quarry, 10Documentation: update quarry page - https://phabricator.wikimedia.org/T311404 (10rook) [13:32:06] awight: btullis: I also think migrating the job to be hourly is possible. We would probably have to create another job to compact the hourly data into daily data, for more efficient storage in Druid, though. [13:32:30] awight how urgent is this? We are migrating all our jobs from Oozie to Airflow currently. And we could do both things at the same time if needed. [13:32:34] 10Quarry, 10Documentation-Review-Board, 10Key docs update 2021-22: Quarry docs - https://phabricator.wikimedia.org/T307011 (10Aklapper) [13:33:22] 10Quarry, 10Documentation-Review-Board, 10Key docs update 2021-22: Quarry docs - https://phabricator.wikimedia.org/T307011 (10Aklapper) Copying in @rook's comment from T311404: > Has some old docs, for instance it has `USE metawiki_p;` though use no longer goes inside of a query in quarry. This ticket is to... [13:34:53] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q4: rack/setup/install stat1010 - https://phabricator.wikimedia.org/T307399 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by btullis@cumin1001 for host stat1010.eqiad.wmnet with OS bullseye [13:37:47] 10Quarry: Quarry database name - https://phabricator.wikimedia.org/T311350 (10Certes) >>! In T311350#8029110, @LucasWerkmeister wrote: > Ich finde den Vorschlag, die englischsprachige Wikipedia zur Standard-Wikipedia in Quarry zu erheben, deutlich irritierender. Die spanischsprachige Wikipedia z.B. hat die fünft... [13:39:02] 10Data-Engineering: Documentathon - https://phabricator.wikimedia.org/T311413 (10JArguello-WMF) [13:41:49] 10Data-Engineering: Documentathon - https://phabricator.wikimedia.org/T311413 (10JArguello-WMF) [13:41:51] 10Analytics: Document our level of support of pyspark-based jobs - https://phabricator.wikimedia.org/T274775 (10JArguello-WMF) [13:41:53] 10Analytics, 10Code-Health: Cleanup Maven dependencies in analytics/refinery - https://phabricator.wikimedia.org/T258802 (10JArguello-WMF) [13:41:59] 10Analytics, 10Code-Health: Integrate SonarCloud analysis as part of the analytics refinery builds - https://phabricator.wikimedia.org/T258800 (10JArguello-WMF) [13:42:05] 10Analytics, 10Code-Health: Introduce various static analysis tools to analytics/refinery - https://phabricator.wikimedia.org/T258699 (10JArguello-WMF) [13:42:09] 10Analytics, 10Documentation: Update Mediawiki Table manuals on wiki (Manual:Logging_table et al) - https://phabricator.wikimedia.org/T179407 (10JArguello-WMF) [13:44:09] 10Data-Engineering-Kanban, 10DSE-Kubernetes-Cluster: Determine IP ranges for dse-k8s cluster - https://phabricator.wikimedia.org/T310169 (10BTullis) 05Open→03Resolved [13:44:23] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Update branding for DataHub to include WMF customizations - https://phabricator.wikimedia.org/T310629 (10BTullis) 05Open→03Resolved [13:44:37] 10Data-Engineering, 10Data-Engineering-Kanban: aqs1008.mgmt interface SSH check flapping - https://phabricator.wikimedia.org/T311042 (10BTullis) 05Open→03Resolved [13:44:53] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Upgrade DataHub V0.8.38 - https://phabricator.wikimedia.org/T310079 (10BTullis) 05Open→03Resolved [13:45:18] mforns: Thanks, I'll ask and will post to the ticket. [13:45:27] cool! [13:46:43] joal: o/ [13:46:57] Hi ottomata [13:47:06] hoooowowwzsszzziiit goinnNnNNNnNGNGNGGNGNGN [13:47:16] All good so far today :) [13:47:27] ottomata: I haz stats (from a minute ago_) [13:47:34] I WAS WUNDERINNGGGG [13:47:37] ottomata: Now I need to analyze data [13:47:41] okkay [13:47:43] cool [13:47:45] ! [13:49:44] I am also interested in the results of this analysis, because I feel a bit concerned about the state of hadoop right now, being unable to fail back cleanly without potential data loss for gobblin. [13:52:09] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q4: rack/setup/install stat1010 - https://phabricator.wikimedia.org/T307399 (10BTullis) [13:52:53] for gobblin, we should just stop it if we need to restart of fail back [13:52:54] for now [13:53:19] the analysis jo al is doing is about where the most small files are, to see if there is any short term mitigation compaction we can do [13:53:24] gobblin, i dunno what is wrong with it [13:54:36] 10Quarry: Quarry database name - https://phabricator.wikimedia.org/T311350 (10rook) I would agree, that having no default, but allowing each user to choose a default for themselves seems appropriate. The home wiki idea is interesting as well. Indeed we could peruse both. Such that the home wiki is the default fo... [13:56:50] OK, thanks for the update. [13:56:53] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q4: rack/setup/install stat1010 - https://phabricator.wikimedia.org/T307399 (10BTullis) I've attempted to run the cookbook to install this server, but it's failing at the TFTP step, I believe. {F35280906} The preceding parts of the cookbook appeared to wor... [13:57:26] 10Data-Engineering: Check home/HDFS leftovers of dsharpe - https://phabricator.wikimedia.org/T310463 (10Ottomata) 05Open→03Resolved a:03Ottomata ` ====== stat1004 ====== total 0 ====== stat1005 ====== total 0 ====== stat1006 ====== total 0 ====== stat1007 ====== total 4 -rw-rw-r-- 1 20828 wikidev 0... [13:57:34] 10Data-Engineering: Check home/HDFS leftovers of dsharpe - https://phabricator.wikimedia.org/T310463 (10Ottomata) 05Resolved→03Open [14:07:29] hi! [14:08:31] Hi milimetric. Welcome back. :-) [14:08:40] thanks btullis, I missed yall! [14:09:09] the "lots of small files" on HDFS issue is getting ugly? [14:11:51] (03PS1) 10Ottomata: Suppress useless GeocodeDatabaseReader log warn messages about 127.0.0.1 not found in database [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/808905 [14:16:32] milimetric: yes, I think so. Seems to be the underlying reason why failing over namenodes is unreliable at the moment. During failover attempts gobblin is also having trouble writing some files, resulting in potential data loss. [14:17:21] Unnerving, to say the least. [14:17:24] ouch. Ok. Well, we knew this day was coming, so we have thoughts, I'll wait to see what's already been discussed [14:21:10] 10Data-Engineering-Icebox: test task - https://phabricator.wikimedia.org/T311418 (10EChetty) [14:25:19] 10Data-Engineering (Sprint 1), 10Equity-Landscape: Readership input metrics - https://phabricator.wikimedia.org/T309273 (10EChetty) [14:25:21] 10Data-Engineering (Sprint 1), 10Equity-Landscape: Affiliates input metric - https://phabricator.wikimedia.org/T309275 (10EChetty) [14:25:37] 10Data-Engineering, 10Equity-Landscape: Affiliates input metric - https://phabricator.wikimedia.org/T309275 (10EChetty) [14:25:47] 10Data-Engineering, 10Equity-Landscape: Readership input metrics - https://phabricator.wikimedia.org/T309273 (10EChetty) [14:31:08] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q4: rack/setup/install stat1010 - https://phabricator.wikimedia.org/T307399 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by btullis@cumin1001 for host stat1010.eqiad.wmnet with OS bullseye executed with errors: - stat1010 (**FAIL**)... [14:37:14] 10Data-Engineering-Icebox: test task - https://phabricator.wikimedia.org/T311418 (10EChetty) 05Open→03Invalid p:05Triage→03Lowest [14:41:05] 10Data-Engineering: Event Utilities partially downloads schemas - https://phabricator.wikimedia.org/T309717 (10Ottomata) I'm starting to regret our old decision to maintain both json and yaml schema formats in the schema repos. (See also {T308450}). The intention was to prefer yaml, but make it a little easier... [14:41:21] 10Data-Engineering: Event Utilities partially downloads schemas - https://phabricator.wikimedia.org/T309717 (10Ottomata) cc @JAllemandou [14:56:39] Hi btullis, could you install this deb in test cluster please ? https://gitlab.wikimedia.org/repos/data-engineering/conda-base-env/-/packages/154 this 0.0.4 (0.0.4.dev !) [15:06:04] ottomata: ping standup? [15:10:59] OH MY [15:25:20] !log upgraded conda-base-env on an-test-client1001 from 0.0.1 to 0.0.4 [15:25:22] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:25:27] aqu: That's done now. [15:25:45] Thx a lot! [15:33:46] 10Data-Engineering, 10Data-Engineering-Kanban: Gobblin dataloss during namenode failure - https://phabricator.wikimedia.org/T311263 (10JArguello-WMF) p:05Triage→03High [15:59:47] 10Analytics, 10API Platform: Establish testing procedure for Druid-based endpoints - https://phabricator.wikimedia.org/T311190 (10BPirkle) Status update: Fiddled around quite a bit with the docker compose solution. Relevant urls: - https://druid.apache.org/docs/latest/tutorials/docker.html - https://druid.apa... [16:53:12] 10Analytics-Radar, 10Anti-Harassment, 10CheckUser, 10Privacy Engineering, and 5 others: Deal with Google Chrome User-Agent deprecation - https://phabricator.wikimedia.org/T242825 (10CBogen) [17:50:04] 10Data-Engineering, 10Release-Engineering-Team, 10GitLab (CI & Job Runners), 10Performance Issue: Improve speed of Gitlab CI - https://phabricator.wikimedia.org/T311111 (10brennen) [18:09:17] 10Data-Engineering, 10Event-Platform: [BUG] jsonschema-tools materializes fields in yaml in a different order than in json files - https://phabricator.wikimedia.org/T308450 (10Ottomata) MR here: https://github.com/wikimedia/jsonschema-tools/pull/40/commits/241b953fda7d66080db4e261b5222f0a3b270465 [18:13:37] (03PS1) 10Ottomata: Spark JsonSchemaConverter - log when schema does not contain type field [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/808978 (https://phabricator.wikimedia.org/T309717) [19:55:35] heads-up, we'll probably see a VarnishKafkaNoMessages alert soon, because someone is depooling and shutting down cp5012. I haven't yet finished T300246 which will fix these false positives. [19:55:36] T300246: Add alert for varnishkafka low/zero messages per second to alertmanager - https://phabricator.wikimedia.org/T300246 [20:06:34] PROBLEM - Check systemd state on stat1005 is CRITICAL: CRITICAL - degraded: The following units failed: jupyter-aarora-singleuser.service,jupyter-seddon-singleuser.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [20:13:28] PROBLEM - Check systemd state on stat1005 is CRITICAL: CRITICAL - degraded: The following units failed: jupyter-aarora-singleuser.service,jupyter-seddon-singleuser.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [20:16:11] The VarnishKafka alert was silenced by sukh.e which was nice. [20:16:13] https://usercontent.irccloud-cdn.com/file/8XRV5lCM/image.png [20:16:45] !log checking and restarting prometheus-mysqld-exporter on an-coord1001 [20:16:46] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:18:52] aqu: I didn't notice that there were errors earlier from the conda-base-env version 0.0.4 package on an-test-client1001. [20:18:58] https://www.irccloud.com/pastebin/plftkNLP/ [20:20:20] PROBLEM - Check systemd state on stat1005 is CRITICAL: CRITICAL - degraded: The following units failed: jupyter-aarora-singleuser.service,jupyter-seddon-singleuser.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [20:20:58] btullis: I was waiting for some puppet propagations. :) Thanks for this error I've never met in dev. [20:25:12] It looks like we've just had a big spike in CPU usage and load on stat1005. It's coming back now, but a couple of user's jupyter sessions were killed. [20:26:08] https://usercontent.irccloud-cdn.com/file/hfFNFnOi/image.png [20:33:40] !log systemctl reset-failed jupyter-aarora-singleuser and jupyter-seddon-singleuser on stat1005 [20:33:41] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:56:44] RECOVERY - Check systemd state on stat1005 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [21:01:42] 10Data-Engineering, 10Data-Engineering-Kanban, 10Event-Platform: [BUG] jsonschema-tools materializes fields in yaml in a different order than in json files - https://phabricator.wikimedia.org/T308450 (10Ottomata) p:05Low→03Medium a:03Ottomata [21:02:45] (03PS1) 10Ottomata: Bump to jsonschema-tools 0.11.0 to get consistent json and yaml serialization ordering [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/809019 (https://phabricator.wikimedia.org/T308450) [21:03:15] btullis: thanks [21:07:04] (03PS1) 10Ottomata: Bump to jsonschema-tools 0.11.0 to get consistent json and yaml serialization ordering [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/809021 (https://phabricator.wikimedia.org/T308450) [21:08:20] (03PS2) 10Ottomata: Bump to jsonschema-tools 0.11.0 to get consistent json and yaml serialization ordering [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/809019 (https://phabricator.wikimedia.org/T308450) [21:08:58] (03CR) 10CI reject: [V: 04-1] Bump to jsonschema-tools 0.11.0 to get consistent json and yaml serialization ordering [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/809019 (https://phabricator.wikimedia.org/T308450) (owner: 10Ottomata) [21:09:41] (03PS3) 10Ottomata: Bump to jsonschema-tools 0.11.0 to get consistent json and yaml serialization ordering [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/809019 (https://phabricator.wikimedia.org/T308450) [21:10:34] (03CR) 10CI reject: [V: 04-1] Bump to jsonschema-tools 0.11.0 to get consistent json and yaml serialization ordering [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/809019 (https://phabricator.wikimedia.org/T308450) (owner: 10Ottomata) [21:13:05] (03PS4) 10Ottomata: Bump to jsonschema-tools 0.11.0 to get consistent json and yaml serialization ordering [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/809019 (https://phabricator.wikimedia.org/T308450)