[00:04:05] PROBLEM - Check unit status of drop_event on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit drop_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [06:22:47] 10Analytics: labstore1006 possible kerberos issue - https://phabricator.wikimedia.org/T284261 (10elukey) @Bstorm that is a motd additional message added by Analytics to hadoop client node, to remember people to kinit if they don't have their credential cache fresh. The labstore nodes are a corner case and they g... [06:34:07] Good morning [06:37:35] bonjour [07:22:10] 10Analytics, 10Wikidata, 10Wikidata-Query-Service: Automate regular WDQS query parsing and data-extraction - https://phabricator.wikimedia.org/T273854 (10AKhatun_WMF) a:03AKhatun_WMF [11:20:27] 10Analytics, 10Analytics-Kanban: Crunch and delete many old dumps logs - https://phabricator.wikimedia.org/T280678 (10Addshore) regarding the dumps logs that I got sucked in Right now this data is still used for this dashboard https://grafana.wikimedia.org/d/000000264/wikidata-dump-downloads?orgId=1&refresh=5m... [11:32:39] 10Analytics, 10Analytics-Wikistats: "Page views by edition of Wikipedia" for each country - https://phabricator.wikimedia.org/T257071 (10A455bcd9) Perfect, I've just created this feature request ( T284294 ). [12:13:27] 10Analytics, 10Analytics-Wikistats: Offer map and queries by subdivisions of countries - https://phabricator.wikimedia.org/T284294 (10Aklapper) Hi @A455bcd9, thanks for taking the time to report this. I assume this is about //stats.wikimedia.org// (please follow https://www.mediawiki.org/wiki/How_to_report_a_b... [12:32:20] 10Analytics, 10Analytics-Wikistats: Offer map and queries by subdivisions of countries - https://phabricator.wikimedia.org/T284294 (10A455bcd9) [12:33:15] 10Analytics, 10Analytics-Wikistats: Offer map and queries by subdivisions of countries - https://phabricator.wikimedia.org/T284294 (10A455bcd9) [12:36:27] 10Analytics, 10Analytics-Wikistats: Offer map and queries by subdivisions of countries - https://phabricator.wikimedia.org/T284294 (10A455bcd9) [12:38:15] 10Analytics, 10Analytics-Wikistats: Offer map and queries by subdivisions of countries - https://phabricator.wikimedia.org/T284294 (10A455bcd9) Hi @Aklapper, yes, I've just added a link to Wikimedia Statistics in my post to make it clearer (and I mentioned the Wikipedia REST API as well). I tried to follow thi... [13:46:25] that's so crazy that AirFlow was developed around the same time as Wikimetrics and shares so much of the basic architecture and requirements [13:46:33] and sad that Wikimetrics went so wrong... [15:36:07] ottomata and joal never play games :) [15:36:08] a-team, someone wanna join the gamezzzzz??!!!?!?!? [15:47:38] Heya team [15:47:42] joining after kids! [16:32:37] Hi razzi - May I ask for an ops task now? [17:07:18] Yes sorry joal didn’t see the notification [17:11:40] 10Analytics, 10Data-release, 10Privacy Engineering, 10Research, 10Privacy: Apache Beam go prototype code for DP evaluation - https://phabricator.wikimedia.org/T280385 (10Htriedman) Thanks @Isaac and @Nuria for the in-depth discussion of the relative pros and cons of these two approaches, and for the deep... [17:12:35] Heya razzi :) [17:13:04] razzi: Can you do https://wikitech.wikimedia.org/wiki/Analytics/Systems/AQS#Deploy_new_History_snapshot_for_Wikistats_Backend ? [17:13:20] razzi: the new snapshot is ready (we missed last month one :S) [17:14:47] 10Analytics-Clusters, 10DC-Ops, 10SRE, 10ops-eqiad: (Need By: TBD) rack/setup/install an-web1001 - https://phabricator.wikimedia.org/T281787 (10Jclark-ctr) an-web1001 B1 U27 Port9 id#3025 [17:15:13] 10Analytics-Clusters, 10DC-Ops, 10SRE, 10ops-eqiad: (Need By: TBD) rack/setup/install an-web1001 - https://phabricator.wikimedia.org/T281787 (10Jclark-ctr) [17:15:59] ah yes, will do that now joal, we should automate that imo [17:16:20] razzi: IIRC there is a cookbook for that [17:16:34] I mean even the scheduling of it [17:16:40] for sure [17:17:19] razzi: I'd even go one step further - the deploy mechanism shouldn't involve a patch [17:17:27] yeah that's what I'm thinking [17:18:27] 10Analytics-Kanban: Analytics Hardware for Fiscal Year 2020/2021 - https://phabricator.wikimedia.org/T255145 (10Jclark-ctr) [17:18:49] 10Analytics-Clusters, 10DC-Ops, 10SRE, 10ops-eqiad: (Need By: TBD) rack/setup/install an-web1001 - https://phabricator.wikimedia.org/T281787 (10Jclark-ctr) a:05Jclark-ctr→03Cmjohnson [17:42:44] !log sudo cookbook sre.aqs.roll-restart aqs to deploy new mediawiki history snapshot [17:42:47] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:16:17] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Generalize the current Airflow puppet/scap code to deploy a dedicated Analytics instance - https://phabricator.wikimedia.org/T272973 (10mforns) Oh, @Ottomata, I think that was the code that we wrote to try and reproduce the error. We could not, meaning thi... [18:18:59] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: New Wikivoyages are only partially included in Stats - https://phabricator.wikimedia.org/T279564 (10razzi) @KuboF this is fixed now, there was an unrelated issue (for those interested, https://wikitech.wikimedia.org/wiki/Analytics/Systems/AQS#Deploy_new... [18:24:44] (03CR) 10Jgreen: "> Patch Set 1:" [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/697972 (owner: 10Jgreen) [18:25:22] (03Abandoned) 10Jgreen: fix typo in superset_database_exists.py [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/697972 (owner: 10Jgreen) [18:35:37] 10Analytics, 10Data-release, 10Privacy Engineering, 10Research, 10Privacy: Apache Beam go prototype code for DP evaluation - https://phabricator.wikimedia.org/T280385 (10TedTed) Thanks @Htriedman for the deep dive. A few notes on our tooling, in case this helps: - Privacy on Beam is still marked experim... [18:52:52] razzi: heya, I didn't check before you launched the cookbook - Have you merged the patch changing the AQS datasource? [18:53:33] joal: yes, self +2’d, the new data is live! [18:53:38] \o/ [18:53:41] awesome [19:08:34] 10Analytics, 10SRE, 10SRE-Access-Requests, 10Patch-For-Review: Requesting access to production shell groups for JAnstee - https://phabricator.wikimedia.org/T266249 (10Dzahn) a:05JAnstee_WMF→03colewhite [21:06:19] 10Analytics, 10SRE, 10SRE-Access-Requests: Requesting access to production shell groups for JAnstee - https://phabricator.wikimedia.org/T266249 (10colewhite) Key has been updated. Please let us know if this action resolved the problem. [21:12:44] 10Analytics, 10Data-release, 10Privacy Engineering, 10Research, 10Privacy: Apache Beam go prototype code for DP evaluation - https://phabricator.wikimedia.org/T280385 (10Nuria) > It might also be possible to do a differentially-private count on a single machine within the Analytics cluster (likely on sta... [21:35:54] 10Analytics, 10Data-release, 10Privacy Engineering, 10Research, 10Privacy: Apache Beam go prototype code for DP evaluation - https://phabricator.wikimedia.org/T280385 (10Htriedman) > Part of this task is to make data releases of this type part of the cycle of data releases at WMF so I do not think we sho... [23:54:17] 10Analytics, 10SRE, 10SRE-Access-Requests: Requesting access to production shell groups for JAnstee - https://phabricator.wikimedia.org/T266249 (10Dzahn) a:05colewhite→03JAnstee_WMF