[01:25:28] PROBLEM - Check unit status of monitor_refine_event on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit monitor_refine_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [09:22:40] (03CR) 10David Caro: [C: 03+1] "Tested that https://quarry.wmcloud.org/DCaro_(WMF) and https://quarry.wmcloud.org/DCaro%20(WMF) go to the same page." [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/771010 (https://phabricator.wikimedia.org/T72166) (owner: 10Vivian Rook) [09:56:13] 10Data-Engineering-Radar, 10Growth-Team, 10MediaWiki-extensions-GuidedTour, 10Patch-For-Review: Finish decommissioning the legacy GuidedTour schemas - https://phabricator.wikimedia.org/T303712 (10kostajh) Thanks for filing this @phuedx and for the patches. Do you need anything from Growth team on this? [10:26:57] !log rerunning failed mediawiki_structured_task_article_link_suggestion_interaction refnie job [10:26:59] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:34:16] 10Data-Engineering: Check home/HDFS leftovers of ppchelko - https://phabricator.wikimedia.org/T303924 (10MoritzMuehlenhoff) [10:38:58] 10Data-Engineering, 10Data-Engineering-Kanban, 10ContentTranslation, 10Language-analytics, 10Product-Analytics: Abuse filter analytics dashboard is broken - https://phabricator.wikimedia.org/T302970 (10MNeisler) @Ottomata I was not involved in the creation of this dataset so I'm not sure. @nshahquinn-... [10:40:03] 10Data-Engineering, 10Data-Engineering-Kanban: Check home/HDFS leftovers of ppchelko - https://phabricator.wikimedia.org/T303924 (10BTullis) p:05Triage→03Medium a:03BTullis [10:48:34] 10Data-Engineering: Check home/HDFS leftovers of accraze - https://phabricator.wikimedia.org/T303928 (10MoritzMuehlenhoff) [11:07:16] 10Data-Engineering, 10Data-Engineering-Kanban: Check home/HDFS leftovers of accraze - https://phabricator.wikimedia.org/T303928 (10BTullis) p:05Triage→03Medium a:03BTullis [11:13:22] RECOVERY - Check unit status of monitor_refine_event on an-launcher1002 is OK: OK: Status of the systemd unit monitor_refine_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [11:55:08] (03CR) 10Vivian Rook: [C: 03+2] Replace space with underscore in link [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/771010 (https://phabricator.wikimedia.org/T72166) (owner: 10Vivian Rook) [11:59:31] (03Merged) 10jenkins-bot: Replace space with underscore in link [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/771010 (https://phabricator.wikimedia.org/T72166) (owner: 10Vivian Rook) [12:06:27] 10Quarry: Make query URLs have a sluggified version of the title in them - https://phabricator.wikimedia.org/T75885 (10rook) [12:06:31] 10Quarry, 10Patch-For-Review: Replace spaces with underscores for wiki usernames in URL - https://phabricator.wikimedia.org/T72166 (10rook) 05Open→03Resolved [12:50:32] 10Analytics-Radar, 10Data-Engineering-Radar, 10Event-Platform, 10MediaWiki-Recent-changes, and 2 others: Remove deprecated RCFeedEngine support - https://phabricator.wikimedia.org/T250628 (10Krinkle) [12:58:15] btullis: FYI am looking into latest refine failure for mediawiki_structured_task_article_link_suggestion_interaction [13:00:20] OH those are the ones you did [13:01:50] hmm, they look refined as of 20 minutes ago [13:01:57] did you rerun day=16/hour=8? [13:03:27] ottomata: yep, I think so. The first rerun didn't work because it said refine_event was already running. [13:03:27] The second time it worked and that was the command that Is copied into the email. [13:03:47] I'm afk at the moment, but back shortly. [13:04:34] I haven't worked out what to do with the mediawiki_drop_snapshot either, so that's still pending and alerting. [13:05:12] btullis: day=16/hour=8 is a different hour than the one you already emailed for [13:05:15] but ya no hury! [13:05:19] respond when around [13:07:38] Ah, ok thanks. [13:08:20] (03PS1) 10Vivian Rook: Update home to direct to profile [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/771352 (https://phabricator.wikimedia.org/T85175) [13:23:09] ottomata: Just checked and I'm fairly sure that I didn't re-run anything for day=16/hour=8 [13:24:31] k thx [13:24:39] ¯\_(ツ)_/¯ [13:25:19] Yeah, also confirm that the files are present with a timestamp of 12:59 [14:01:19] heya ottomata - you left 2 comments unanswred in the gobblin PR - they are minor, but it'd be great to resolve them one way of another :) [14:01:27] (03CR) 10Joal: Add PrometheusEventReporter (033 comments) [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/767178 (owner: 10Joal) [14:02:15] oh [14:19:33] 10Analytics-Radar, 10Data-Engineering-Radar, 10Event-Platform, 10MediaWiki-Recent-changes, and 2 others: Remove deprecated RCFeedEngine support - https://phabricator.wikimedia.org/T250628 (10Majavah) [14:56:04] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Persistence (Consultation), 10Data-Services, and 2 others: Toolforge db: View 'fiwiki_p.flaggedrevs' references invalid table/column/rights to use them - https://phabricator.wikimedia.org/T302233 (10Marostegui) What's the status of this? This is causing... [15:11:48] 10Data-Engineering, 10Data-Catalog, 10SRE, 10serviceops, and 2 others: New Service Request: DataHub - https://phabricator.wikimedia.org/T303049 (10BTullis) I have created deployment users and tokens in `profile::kubernetes::infrastructure_users:` key in the private repo, as well as corresponding dummy valu... [15:16:13] 10Data-Engineering, 10Data-Engineering-Kanban, 10ContentTranslation, 10Language-analytics, 10Product-Analytics: Abuse filter analytics dashboard is broken - https://phabricator.wikimedia.org/T302970 (10nshahquinn-wmf) > @nshahquinn-wmf or @Amire80 Do you know the source of the amire80.cx_abuse_filter_dai... [15:36:29] joal oh sorry i missed those, if i fix those after meetings, you okay with me merging and getting deployment patches in order? [15:44:05] (03CR) 10Aqu: [V: 03+2 C: 03+2] "- Add Airflow session length HQL file" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/769515 (https://phabricator.wikimedia.org/T300029) (owner: 10Aqu) [16:03:00] !log analytics/refinery - scap deply "Migrate session_length/daily from Oozie to Airflow" [16:03:02] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:30:26] will be in SRE sync in 1 min [16:32:30] 10Data-Engineering, 10SRE, 10ops-eqiad: analytics10[63,67] mgmt interfaces seem flapping from time to time - https://phabricator.wikimedia.org/T303151 (10Cmjohnson) @BTullis Can you plan to shut this down tomorrow 17 March at 10a EST 1400 UTC. [16:33:16] 10Data-Engineering, 10SRE, 10ops-eqiad: analytics10[63,67] mgmt interfaces seem flapping from time to time - https://phabricator.wikimedia.org/T303151 (10BTullis) Yes, will do. Both nodes at the same time? [16:49:38] 10Data-Engineering, 10Data-Engineering-Kanban: Resume Webrequest Data Purge Job - https://phabricator.wikimedia.org/T303977 (10odimitrijevic) [16:51:24] 10Data-Engineering, 10Data-Engineering-Kanban: Resume Webrequest Data Purge Job - https://phabricator.wikimedia.org/T303977 (10odimitrijevic) p:05Triage→03High [17:01:02] ottomata: yesiir - let's deploy that thing :) [17:01:42] ottomata: actually, we need quite a mess for the deploy: we need to move from old-gobblin-repo to new one - no major changes, but still, some tests [17:28:19] rigiht [17:28:51] oh and we need https://gerrit.wikimedia.org/r/c/analytics/gobblin-wmf/+/761400 [17:28:52] ? [17:29:06] joal: Can we catch up tomorrow morning about the mediawiki_drop_snapshots change? [17:29:09] ottomata: it'd be great [17:29:37] btullis: yes - many meetings tomorrow morning, but we'll find a spot :) [17:29:43] btullis: can also be now if ou wish [17:30:10] (03PS13) 10Ottomata: Add PrometheusEventReporter [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/767178 (owner: 10Joal) [17:30:15] (03CR) 10Ottomata: Add PrometheusEventReporter (032 comments) [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/767178 (owner: 10Joal) [17:30:15] Aren't you deploying the gobblin thing now? ;-) [17:30:25] now not deploying now, i'm just getting things in order [17:30:45] btullis: I mostly am, but I'm with Andrew - I'm barely needed ;) [17:31:04] OK, I'm game if you are. Batcave? [17:31:21] joal i'm going to merge some things and then try to get patches in order [17:31:23] (03CR) 10Ottomata: [C: 03+2] Update Gobblin version to 0.16 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/761400 (owner: 10Joal) [17:31:29] ack ottomata [17:31:36] (03PS14) 10Ottomata: Add PrometheusEventReporter [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/767178 (owner: 10Joal) [17:32:13] ottomata: we need to test the gobblin 16 thing as well [17:33:40] ok joal i'll just use the same test with rebased jar [17:33:44] that i've been using [17:33:52] great ottomata [17:33:57] btullis: batcave? [17:34:03] Yup. [17:34:06] 10Data-Engineering, 10Data-Engineering-Kanban, 10Airflow, 10Patch-For-Review: Low Risk Oozie Migration: session length - https://phabricator.wikimedia.org/T300029 (10Antoine_Quhen) https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/33 [17:34:42] (03CR) 10jerkins-bot: [V: 04-1] Add PrometheusEventReporter [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/767178 (owner: 10Joal) [17:34:43] joal: to build/release, will i be manually uploading the gobblin-wmf-shaded jar to archiva? [17:35:23] hmm [17:35:23] https://archiva.wikimedia.org/#artifact/org.wikimedia.gobblin/gobblin-wmf [17:35:36] do i need to setup maven release plugin? [17:35:43] ottomata: now we have maven I think we have a release job [17:35:50] in jenkins? [17:35:53] or we can upload manually [17:35:56] yeah! [17:35:57] or from CLI [17:35:57] ? [17:35:59] oh! [17:36:01] for gobblin-wmf!? [17:36:02] let's check [17:44:04] (03CR) 10Ottomata: [V: 03+2 C: 03+2] "CI fails because of javadoc issues in copy.* classes. These will be removed after upstream fixes their versions and we can just use their" [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/767178 (owner: 10Joal) [17:50:13] ottomata: if CI fails it means we can 't have jenkins dpeloying :( [17:50:23] 10Data-Engineering: Refactor refinery-drop-mediawiki-snapshots to not use _SUCCESS file - https://phabricator.wikimedia.org/T303988 (10BTullis) [17:50:32] OHhh, but CI always fails because of copy files [17:50:35] okay i can just fix [17:50:36] them [17:50:37] that okay? [17:50:56] sure! [17:51:03] that's not great though [17:51:12] that we have to adapt those things [17:51:17] yeah [17:51:22] 13:46:43 Exit code: 1 - /src/gobblin-wmf-core/src/main/java/org/wikimedia/gobblin/copy/BaseDataPublisher.java:96: error: unexpected end tag:

[17:51:42] https://integration.wikimedia.org/ci/job/analytics-gobblin-wmf-maven-java8-docker/65/console [17:52:00] joal maybe i'll just remove the javadoc comments? [17:52:27] sure ottomata - let's add a comment about that removal and I'm fine with it [17:52:31] okay [17:53:35] hey ottomata and anyone else that can merge puppet: https://gerrit.wikimedia.org/r/c/operations/puppet/+/771389 [17:53:53] (re-enabling webrequest / pageview_actor purge) [17:55:15] milimetric: I'm waiting for a confirmation from Erin - would you mind waiting a bit? [17:55:23] (03PS1) 10Ottomata: Remove javadoc in gobblin upstream copied files to make javadoc generating CI happy [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771418 [17:55:37] joal: olja said to go ahead in slack... I'll check with her again [17:55:45] milimetric: I did! [17:55:52] milimetric: please ping her again :) [17:56:08] -1ing until then [17:56:23] thank you :) [17:56:24] heh, oh Jenkins didn't like it anyway [17:56:30] arf [17:56:38] jenkins is picky [17:57:54] !log restarted mediawiki-history-drop-snapshot service on an-launcher1002 [17:57:56] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [17:58:18] (03CR) 10jerkins-bot: [V: 04-1] Remove javadoc in gobblin upstream copied files to make javadoc generating CI happy [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771418 (owner: 10Ottomata) [17:58:25] RECOVERY - Check unit status of mediawiki-history-drop-snapshot on an-launcher1002 is OK: OK: Status of the systemd unit mediawiki-history-drop-snapshot https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [17:59:36] 10Analytics-Wikistats, 10Data-Engineering: Broken "tooltip-breakdown-automated" tooltip on Wikistats 2 - https://phabricator.wikimedia.org/T303990 (10Krinkle) [18:00:38] !log sudo cookbook sre.hosts.downtime -D 3 -r 'Setting up karapace for the first time' karapace1001.eqiad.wmnet [18:00:40] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:00:59] Downtimed karapace until the end of the week, so it won't alert as I start to configure it [18:02:55] 10Data-Engineering-Radar, 10Growth-Team, 10MediaWiki-extensions-GuidedTour, 10Patch-For-Review: Finish decommissioning the legacy GuidedTour schemas - https://phabricator.wikimedia.org/T303712 (10phuedx) >>! In T303712#7781001, @kostajh wrote: > Thanks for filing this @phuedx and for the patches. Do you ne... [18:05:09] We made a ticket to fix the mediawiki-history-drop-snapshots task: https://phabricator.wikimedia.org/T303988 [18:05:37] In the short term we just touched the _SUCCESS files manually and restarted the job. [18:06:10] thanks again btullis --^ [18:06:20] A pleasure. [18:06:21] (03PS2) 10Ottomata: Remove javadoc in gobblin upstream copied files to make javadoc generating CI happy [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771418 [18:06:39] (03PS3) 10Ottomata: Remove javadoc in gobblin upstream copied files to make javadoc generating CI happy [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771418 [18:08:31] 10Data-Engineering: Add the commons-entity dataset to the refinery-drop-mediawiki-snapshots script - https://phabricator.wikimedia.org/T303993 (10JAllemandou) [18:10:05] (03CR) 10Ottomata: [C: 03+2] Remove javadoc in gobblin upstream copied files to make javadoc generating CI happy [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771418 (owner: 10Ottomata) [18:10:20] okay joal CI all happy [18:10:23] https://gerrit.wikimedia.org/r/c/analytics/gobblin-wmf/+/771418 [18:10:38] ottomata: and tested with gobblin 16? [18:10:41] yes [18:10:44] \o/ [18:10:46] i ran my job at least [18:10:47] :) [18:10:47] you rock [18:11:06] just had to change the gobblin.kafka.consumerClient.class and it worked [18:11:09] that CI is so fast I'm almost crying [18:11:30] 4mins? :p [18:11:34] https://integration.wikimedia.org/ci/job/analytics-gobblin-wmf-maven-java8-docker/68/ [18:11:44] oh oyu mean compared to gobblin fork? [18:11:51] or even refinery [18:11:55] aye [18:11:59] okay sooOo how to release? [18:12:05] checking [18:12:19] guess we got to update https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin#Releasing_new_Gobblin_versions ? [18:12:36] very much so - gobblin docs will need to be changed [18:12:40] kay [18:15:09] ottomata: we have https://integration.wikimedia.org/ci/job/analytics-gobblin-wmf-maven-release-docker/build [18:15:19] and I assume this should its magic [18:15:28] now the thing to consider is versioning [18:15:31] ottomata: --^ [18:16:06] OOOO [18:16:07] okay [18:16:12] ottomata: we'll have gobblin-wmf-1.0 if we run it as is [18:16:18] lets go. i will try? [18:16:33] shall we go with that or something else? [18:16:41] naw 1.0 is fine ya? [18:16:48] yeah, works for me [18:16:56] try that thing :) [18:17:04] hm maybe 1.0.0 [18:17:05] eh? [18:17:12] i'll update pom and lets go with thatt [18:17:14] if you prefer [18:17:49] and the next one will be 1.1.0-SNAPSHOT IIRC how maven does its thing [18:17:54] that's fine [18:18:13] (03PS1) 10Ottomata: Set version to 1.0.0-snapshot [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771424 [18:18:21] (03CR) 10Ottomata: [V: 03+2 C: 03+2] Set version to 1.0.0-snapshot [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771424 (owner: 10Ottomata) [18:18:37] building [18:18:43] exciting :) [18:19:19] Starting build #1 for job analytics-gobblin-wmf-maven-release-docker [18:19:31] Project analytics-gobblin-wmf-maven-release-docker build #1: 04FAILURE in 12 sec: https://integration.wikimedia.org/ci/job/analytics-gobblin-wmf-maven-release-docker/1/ [18:20:02] ottomata: we need some patches for jar update - adding the jar to refinery and also some puppet changes for jar version [18:20:46] ya [18:21:07] mwarf :S [18:22:39] oo hm version missing in parent pom mversion i think [18:22:47] refinery-source has verison in one place, right? [18:22:48] looking [18:23:35] ottomata: the numbers must match between project and subproject I think [18:23:41] I was investingating that [18:23:41] yes [18:23:47] and currently they don't :) [18:24:08] gobblin core core references parent with version 1.0 [18:24:16] ya i see [18:24:24] i also thikn i can remove version from gobblin-wmf-core [18:24:26] trying [18:24:52] looks like so from refinery-source [18:26:20] (03PS1) 10Ottomata: Reference parent pom version properly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771425 [18:26:31] okay ^ should fix, was able to mvn package too [18:27:36] (03PS2) 10Ottomata: Reference parent pom version properly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771425 [18:27:47] i'll wait for CI this time :p [18:27:52] huhu :) [18:32:01] (03CR) 10jerkins-bot: [V: 04-1] Reference parent pom version properly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771425 (owner: 10Ottomata) [18:32:54] >:| [18:32:57] 14:31:56 [ERROR] The xml element should be placed before [18:32:57] 14:31:56 [ERROR] The file /src/gobblin-wmf-core/pom.xml is not sorted [18:33:04] there is no ! [18:33:09] it is inherited! [18:33:18] OH wait maybe thatts in parent pom ref [18:33:46] (03PS3) 10Ottomata: Reference parent pom version properly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771425 [18:38:53] (03CR) 10Ottomata: [C: 03+2] Reference parent pom version properly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/771425 (owner: 10Ottomata) [18:39:24] Starting build #2 for job analytics-gobblin-wmf-maven-release-docker [18:47:19] Project analytics-gobblin-wmf-maven-release-docker build #2: 04STILL FAILING in 7 min 55 sec: https://integration.wikimedia.org/ci/job/analytics-gobblin-wmf-maven-release-docker/2/ [18:48:35] hmm [18:48:37] 14:47:18 [ERROR] remote: error: branch refs/heads/main: [18:48:37] 14:47:18 [ERROR] remote: To push into this reference you need 'Push' rights. [18:48:37] 14:47:18 [ERROR] remote: User: maven-release-user [18:50:05] hm [18:50:09] gerrit perms [18:50:11] for jenkins bot i think [18:50:19] copyhing from reinery-source [18:50:35] in gerrit you mean ottomata ? [18:50:38] ya [18:50:42] https://gerrit.wikimedia.org/r/admin/repos/analytics/refinery/source,access [18:50:50] just modified https://gerrit.wikimedia.org/r/admin/repos/analytics/gobblin-wmf,access [18:50:54] trying again [18:51:14] Starting build #3 for job analytics-gobblin-wmf-maven-release-docker [18:52:43] (03CR) 10Milimetric: [C: 03+1] "Oh, you already merged :P" [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/761400 (owner: 10Joal) [18:54:14] milimetric: we fixin up stuff getting ready ^^^ [18:59:35] yeehaw joal https://archiva.wikimedia.org/#artifact/org.wikimedia.gobblin/gobblin-wmf [18:59:41] https://archiva.wikimedia.org/#artifact/org.wikimedia.gobblin/gobblin-wmf/1.0.0 [18:59:53] Yippee, build fixed! [18:59:53] Project analytics-gobblin-wmf-maven-release-docker build #3: 09FIXED in 8 min 38 sec: https://integration.wikimedia.org/ci/job/analytics-gobblin-wmf-maven-release-docker/3/ [19:00:03] \o/ ottomata [19:00:11] let's wait th madatory few minutes :) [19:00:49] :) [19:01:03] i still need to add to refinery artifacts anyway [19:01:50] hmm wait [19:01:52] where is hte jar? [19:01:52] https://archiva.wikimedia.org/#artifact-details-download-content/org.wikimedia.gobblin/gobblin-wmf/1.0.0 [19:02:30] ottomata: https://archiva.wikimedia.org/#artifact/org.wikimedia.gobblin/gobblin-wmf-core [19:02:35] OHOH right [19:02:37] phewf [19:02:39] :) [19:05:51] (03PS1) 10Ottomata: Add gobblin-wmf-core-1.0.0-jar-with-dependencies.jar [analytics/refinery] - 10https://gerrit.wikimedia.org/r/771431 (https://phabricator.wikimedia.org/T297939) [19:06:03] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Update refinery gobblin jars to use new gobblin-wmf jars and update puppet gobblin jobs - https://phabricator.wikimedia.org/T297939 (10Ottomata) a:03Ottomata [19:06:26] (03CR) 10Ottomata: [V: 03+2 C: 03+2] Add gobblin-wmf-core-1.0.0-jar-with-dependencies.jar [analytics/refinery] - 10https://gerrit.wikimedia.org/r/771431 (https://phabricator.wikimedia.org/T297939) (owner: 10Ottomata) [19:07:16] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Add Jenkins job for gobblin-wmf jar release to archiva - https://phabricator.wikimedia.org/T297938 (10Ottomata) a:03JAllemandou [19:07:25] joal i'm going to deploy refinery to test, okay? [19:07:31] also [19:07:32] https://gerrit.wikimedia.org/r/c/operations/puppet/+/771430 [19:07:41] ottomata: shouldn't we also update the symlink in main artifacts folder? [19:07:46] not yet [19:07:51] hmm [19:07:53] actually [19:08:00] yes, maybe, let's deploy my puppet patch first then [19:08:08] it changes the existent jobs to use the old versioned jar [19:08:14] instead of thet symlink [19:10:00] hmm, or maybe just new symlink joal? gobblin-wmf-core.jar? [19:10:25] works for me ottomata - we can remove the other when we're done [19:10:28] k [19:11:52] (03PS1) 10Ottomata: Add gobblin-wmf-core-jar-with-dependencies.jar versionless symlink [analytics/refinery] - 10https://gerrit.wikimedia.org/r/771432 [19:12:27] joal ^ [19:12:27] ? [19:13:07] (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for test" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/771432 (owner: 10Ottomata) [19:13:20] merged ottomata [19:13:22] danke [19:14:13] !log deploying refinery to hadoop-test cluster with new gobblin-wmf-core jar [19:14:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:17:09] oh i should also update the gobblin job conf file in test to use metrics! [19:17:27] OH [19:17:32] and the non copy class [19:18:11] hmmmm oh it doesn't use the copy class right now joal? [19:18:43] ottomata: there was no copy class - it was using the non-forked stuff [19:19:00] okay [19:20:45] ahhh foooo i think we need a hole in the vlan for pushgateway [19:21:11] Ah! /o\ I had not thought about that :( [19:23:33] 10Data-Engineering, 10Data-Engineering-Kanban: Allow access to prometheus-pushgateway.discovery.wmnet port 80 from within Analytics VLAN - https://phabricator.wikimedia.org/T304001 (10Ottomata) [19:24:05] 10Data-Engineering, 10Data-Engineering-Kanban, 10Infrastructure-Foundations, 10netops: Allow access to prometheus-pushgateway.discovery.wmnet port 80 from within Analytics VLAN - https://phabricator.wikimedia.org/T304001 (10Ottomata) [19:26:08] 10Data-Engineering, 10Data-Engineering-Kanban, 10Infrastructure-Foundations, 10netops: Allow access to prometheus-pushgateway.discovery.wmnet port 80 from within Analytics VLAN - https://phabricator.wikimedia.org/T304001 (10Ottomata) [19:26:16] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Send some existing Gobblin metrics to prometheus - https://phabricator.wikimedia.org/T294420 (10Ottomata) [19:32:34] ottomata: I plan on leaving soon for diner - ok for you? [19:39:08] taking this as a yes :) [19:54:01] joal: ya go for it [19:54:15] i'm just going to get as far as i can today, which won't be much farther :) [19:54:19] see ya tomoprrow! [20:14:40] 10Data-Engineering, 10Data-Engineering-Kanban, 10Infrastructure-Foundations, 10SRE, and 2 others: Allow access to prometheus-pushgateway.discovery.wmnet port 80 from within Analytics VLAN - https://phabricator.wikimedia.org/T304001 (10Ottomata) [20:24:03] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Persistence (Consultation), 10Data-Services, and 2 others: Toolforge db: View 'fiwiki_p.flaggedrevs' references invalid table/column/rights to use them - https://phabricator.wikimedia.org/T302233 (10razzi) I have been working on a new cookbook to update... [21:08:20] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Recreate views for globaluser table - https://phabricator.wikimedia.org/T301674 (10Zabe) [21:08:39] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Recreate views for globaluser table - https://phabricator.wikimedia.org/T301674 (10Zabe) [21:08:50] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Recreate views for globaluser table - https://phabricator.wikimedia.org/T301674 (10Zabe) [21:08:59] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Recreate views for globaluser table - https://phabricator.wikimedia.org/T301674 (10Zabe) [21:22:18] (03CR) 10Kosta Harlan: [C: 03+1] Remove mentions of GuidedTour* legacy EventLogging schemas [analytics/refinery] - 10https://gerrit.wikimedia.org/r/770900 (https://phabricator.wikimedia.org/T303712) (owner: 10Phuedx) [22:24:19] 10Data-Engineering, 10Data-Engineering-Kanban, 10Infrastructure-Foundations, 10SRE, and 2 others: Allow access to prometheus-pushgateway.discovery.wmnet port 80 from within Analytics VLAN - https://phabricator.wikimedia.org/T304001 (10cmooney) Worth noting that we are planning in the short term to adjust t...