[00:24:03] 10Fundraising Sprint Overused petting Zoo Memetics, 10Fundraising-Backlog, 10FR-Braintree-Integration: Paypal audit parser should drop Braintree-initiated donations - https://phabricator.wikimedia.org/T315258 (10AnnWF) a:03AnnWF [02:01:40] (03PS2) 10Ejegg: Use ReferenceData to decode Ingenico submethods [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/790788 (https://phabricator.wikimedia.org/T308088) [05:06:30] (03CR) 10CI reject: [V: 04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_38) - 10https://gerrit.wikimedia.org/r/823873 (owner: 10L10n-bot) [07:58:17] (03CR) 10Abijeet Patro: [V: 03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_38) - 10https://gerrit.wikimedia.org/r/823873 (owner: 10L10n-bot) [15:57:07] 10Fundraising Sprint Overused petting Zoo Memetics, 10Fundraising-Backlog, 10FR-Braintree-Integration: Migrate PayPal EC billing agreements to Braintree vault tokens - https://phabricator.wikimedia.org/T315371 (10Cstone) On the new contribution_recur vs updating old ones, we will lose the history of payments... [15:57:47] hi fr-tech i'll be a bit late to the meeting today [16:17:52] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-Alerts: INVALID_MESSAGE Recurring donation, but no subscription ID or recurring payment token found failmail coming from the listener on ideal - https://phabricator.wikimedia.org/T315031 (10Cstone) We need to email adyen to see why we aren't gettin... [16:37:43] 10Fundraising-Backlog, 10Data-Engineering, 10Event-Platform Value Stream, 10MediaWiki-extensions-EventLogging, and 2 others: Determine which remaining legacy EventLogging schemas need to be migrated or decommissioned - https://phabricator.wikimedia.org/T282131 (10phuedx) [17:04:29] (03CR) 10Ejegg: [C: 04-1] "It would be better to do this at the SmashPig level rather than the DonationInterface level. We would be able to leave the DonationInterfa" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/822191 (https://phabricator.wikimedia.org/T312808) (owner: 10Wfan) [17:06:37] you mean add the merchantId directly from PaymentProviders/Braintree/PaypalPaymentProvider.php not from DI right~ ok on it [17:06:43] (03CR) 10Ejegg: [C: 04-1] "Let's do the mapping from currency to merchant account ID here rather than in the calling classes. I picture us adding the account map as " [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/822192 (https://phabricator.wikimedia.org/T312808) (owner: 10Wfan) [17:06:53] wfan yep! [17:07:20] gotcha~ ok [17:07:21] I think we could even do it in a way that wouldn't need any patch at the DI level [17:08:44] how about the currency and decimal from di? [17:09:46] so DI should be able to send the currency code like it does for all the other adapters [17:11:08] and should send the amount in major units (i.e. dollars) [17:11:14] Yeah I mean in this https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/822191/ pr, I modified the currencies yaml and add two extra currencies to decimal amount, those should stay here right~ [17:11:57] oh hmm, new non-fractional currencies? [17:12:07] yep, for paypal [17:12:21] both ec and braintree should be non-fractional [17:12:41] Which braintree is using the same list as paypal does [17:13:29] HUF is marked as (2) 'Exotic currency' in that list, not a (*) Zero-decimal currency [17:14:02] https://docs.adyen.com/development-resources/currency-codes says HUF has 2 decimals [17:14:14] same with TWD [17:14:18] Taken from https://developer.paypal.com/reference/currency-codes/#paypal-account-payments for braintree paypal [17:14:35] this is confirmed by Braintree via email let me forward that email to u [17:14:46] oh, that's strange [17:14:57] OK, so that would be handled down at the SmashPig level [17:15:11] since it differs between different payment providers [17:15:46] The idea is for SmashPig to accept the same parameters and the same formats across all providers [17:16:32] at the braintree reference they are marked differently: [17:16:38] https://developer.paypal.com/braintree/docs/reference/general/currencies [17:17:05] TWD has no star marking zero-decimal nor any other marking [17:17:16] and HUF is marked as exotic rather than zero-decimal [17:17:17] Yeah, but after confirmed with braintree, (you can check the option list form sandbox business account as well) so the currency list is no longer the same as braintree doc indicated. but use the paypal one~ [17:17:37] Email forwarded to you about this zero-decimal discussion [17:18:04] OK, I trust that you have confirmed it. What a pain that the documentation is conflicting! [17:19:11] So if you want to format them differently for display in DonationInterface, you'll need to make a subclass of Amount (I guess called PayPalAmount) and have it override the zero-decimal currency list [17:19:28] because other gateways should still be able to handle fractional amounts of HUF and TWD [17:19:44] ok, make sense~ [17:20:26] Thanks Elliott :) [17:20:36] sure thing! [17:40:48] 10Fundraising-Backlog: Way to send Civi TY email to a group? - https://phabricator.wikimedia.org/T314525 (10MDemosWMF) We are using this contributions [[ https://civicrm.wikimedia.org/civicrm/admin/search#/edit/597 | search kit ]] to pull the contacts who need to be sent a thank you. Could we send an email TY di... [17:43:11] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Search kit question - https://phabricator.wikimedia.org/T315376 (10MDemosWMF) @Eileenmcnaughton Thank you! That set me back on the right track. I also added "if street address is not empty" before the "primary address = yes" and that gave me only contacts... [17:43:51] 10Fundraising-Backlog: Way to send Civi TY email to a group? - https://phabricator.wikimedia.org/T314525 (10MDemosWMF) @Eileenmcnaughton this is a continuation of how we would like to use the search kit from our previous [[ https://phabricator.wikimedia.org/T315376 | ticket ]]. [17:53:55] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fr-donorservices: Civi: underscore in email address slows search - https://phabricator.wikimedia.org/T234100 (10MBeat33) cid=842222 has one underscore in the email address, and the email search time is 1.5 minutes. [17:54:28] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fr-donorservices: Civi: underscore in email address slows search - https://phabricator.wikimedia.org/T234100 (10MBeat33) This Task may be a duplicate of {{T147156}} [17:58:43] 10fundraising-tech-ops: Install and configure new host frlog1002 - https://phabricator.wikimedia.org/T312581 (10Jgreen) a:03Jgreen [17:59:37] 10fundraising-tech-ops: Install and configure new host frlog1002 - https://phabricator.wikimedia.org/T312581 (10Jgreen) [18:02:42] 10fundraising-tech-ops: Install and configure new host frlog1002 - https://phabricator.wikimedia.org/T312581 (10Jgreen) [18:02:45] 10Fundraising-Backlog: Increase country score for Italy - https://phabricator.wikimedia.org/T315468 (10AJett) [18:50:07] 10fundraising-tech-ops: Fundraising access request for Janaina Moreira - https://phabricator.wikimedia.org/T315248 (10Dwisehaupt) [18:50:31] 10fundraising-tech-ops: Fundraising access request for Janaina Moreira - https://phabricator.wikimedia.org/T315248 (10Dwisehaupt) 05Open→03Resolved a:03Dwisehaupt Verified access via log entries. Closing. [19:47:07] hi fr-tech! Reminder that this week’s 3-hour pre-test is launching in 15 minutes :) [19:48:17] thanks haley_ [19:49:31] fr-tech so this is our opportunity to try out this fix that eileen found: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/civicrm/+/820576 [19:49:42] It's not deployed to CRM yet - I'll get that preparted [19:49:47] *prepared [19:51:34] (03PS1) 10Ejegg: Update CiviCRM submodule [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824274 [19:51:41] (03CR) 10Ejegg: [C: 03+2] Update CiviCRM submodule [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824274 (owner: 10Ejegg) [19:51:54] np! Let me know if it'd be helpful to delay the start of the test at all ejegg [19:52:26] haley_: no need to delay - we'll measure the performance with the current code, then deploy that fix and see how it changes [19:52:57] oh ok, gotcha! [19:54:00] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/824278 [19:54:04] (03CR) 10Ejegg: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/824278 (owner: 10Ejegg) [20:08:29] (03Merged) 10jenkins-bot: Update CiviCRM submodule [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824274 (owner: 10Ejegg) [20:16:46] OK, that's ready to go out, but let's wait till we see some significant queue backup [20:29:36] 10Fundraising Sprint Overused petting Zoo Memetics, 10Fundraising-Backlog, 10FR-Braintree-Integration: Migrate PayPal EC billing agreements to Braintree vault tokens - https://phabricator.wikimedia.org/T315371 (10Ejegg) The PayPal engineer on the call today suggests we can get the list of active recurring pa... [20:41:40] 10Fundraising-Backlog, 10fr-donorservices: Donations appearing in Civi but not Adyen - https://phabricator.wikimedia.org/T315487 (10AMJohnson) [21:05:22] hi eileen___ ! [21:05:29] we've got a decent queue backlog worked up [21:05:35] and the patch all ready to deploy [21:05:49] just needs the f_c_u and the rsync from frpm [21:06:07] ejegg: yeah I only just wok up! [21:06:13] let's do it [21:06:27] ok, here we go [21:06:40] yay [21:08:06] !log updated civicrm from c228e3d7 to 4be0724d [21:08:08] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:12:51] well, from the last two data points it's faster after the deploy [21:13:05] oh I can't see that show up yet [21:13:30] but we can add in this patch & try it on top https://github.com/civicrm/civicrm-core/pull/24129/commits/4f0569f6e03b00769c468c6db63fc57d50204fdb [21:13:31] heh, just looking at the 21:10 times [21:13:44] which is actually probably still with the old code now that I think of it [21:13:54] but ^^ was only a smaller change on staging [21:14:20] sounds good eileen [21:14:28] want to give it a half hour with the first patch [21:14:34] before adding the second? [21:14:57] sure - I have to prep it anyway [21:15:09] also - we should prob look at the update contact line [21:15:16] woohoo, new data point is way faster! [21:15:30] yep, that's down near 50ms for the latest measurement [21:15:37] yeah [21:15:38] after consistent ~150ms [21:15:53] it's a clear pic for the last quarter hour now [21:15:55] create is up a little [21:16:06] but it's fluctuated quite a bit [21:16:37] yeah only update would be affected by the patch [21:17:21] oh ok [21:20:33] (03PS1) 10Eileen: Further caching fix - use metadata cache [wikimedia/fundraising/crm/civicrm] - 10https://gerrit.wikimedia.org/r/824295 (https://phabricator.wikimedia.org/T313000) [21:20:49] ejegg: it has hit a new plateau [21:21:07] cool cool, but about back where it was last big english [21:21:23] for update? probably not [21:21:42] ah, a bit slower still [21:22:45] hmm - the mix between create & update will matter [21:23:19] I've put up that second fix https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/civicrm/+/824295 [21:25:04] (03CR) 10CI reject: [V: 04-1] Further caching fix - use metadata cache [wikimedia/fundraising/crm/civicrm] - 10https://gerrit.wikimedia.org/r/824295 (https://phabricator.wikimedia.org/T313000) (owner: 10Eileen) [21:26:56] oh weird [21:26:57] [error_message] => 'Mailing' is not a valid option for field extends [21:27:11] in the omnimail install [21:29:27] ahh, newly cached stuff [21:29:49] maybe not being flushed after the Mailing option was added? [21:31:28] hmm, I don't see the old value being flushed anywhere either [21:31:33] not by specific key anyway [21:33:31] could it be that the metadata cache is lasting across multiple command-line calls in the CI environment, while the previous cache was just lasting one call? [21:35:55] not really from the change made [21:36:25] cos it just improves the memory of cache keys within session [21:39:42] ok, so cache.metadata defaults to *memory*, then falls back to SqlGroup if none of the redis/memcache/ things are available [21:40:07] while the old CRM_Utils_Cache seems to default to ArrayCache, if I'm reading it right [21:40:36] So maybe the post-patch really is caching across requests? [21:42:20] no - it defaults to Redis [21:42:36] Hmm, in OptionValue::Save I see we do a CRM_Core_PseudoConstant::flush(); [21:42:45] but no flushing of any other caches [21:42:53] eileen___: is redis available in CI? [21:42:59] no... [21:46:41] (03PS2) 10Eileen: Further caching fix - use metadata cache [wikimedia/fundraising/crm/civicrm] - 10https://gerrit.wikimedia.org/r/824295 (https://phabricator.wikimedia.org/T313000) [21:47:07] I've added in an extra flush (although it might be the opposite - ie it relies on us flushing badly :-) [21:48:17] that looks correct [21:52:01] lets see what ci thinks [21:57:05] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm/civicrm] - 10https://gerrit.wikimedia.org/r/824295 (https://phabricator.wikimedia.org/T313000) (owner: 10Eileen) [21:57:09] seems to have gotten a lot further already [21:57:20] i.e. it's gotten through the setup and is running the tests [21:57:24] https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicrm-docker/8311/console [21:57:41] I'd be happy to +2 it [21:57:56] (03CR) 10Ejegg: [C: 03+2] "Looks good to me!" [wikimedia/fundraising/crm/civicrm] - 10https://gerrit.wikimedia.org/r/824295 (https://phabricator.wikimedia.org/T313000) (owner: 10Eileen) [21:59:04] 10Fundraising Sprint NaN is a Number, 10Fundraising Sprint Overused petting Zoo Memetics, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Investigate queue consumer slowdown in pretest of July 13 2022 - https://phabricator.wikimedia.org/T313000 (10Eileenmcnaughton) We just dep... [22:00:10] cool [22:03:26] I'll start the rest of the commits to get that deployable [22:04:35] (03PS1) 10Ejegg: Update civicrm submodule [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824304 [22:04:40] (03CR) 10Ejegg: [C: 03+2] Update civicrm submodule [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824304 (owner: 10Ejegg) [22:05:11] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/824305 [22:05:13] (03CR) 10Ejegg: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/824305 (owner: 10Ejegg) [22:08:03] (03PS1) 10Eileen: CiviCRM Submodule udpate [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824307 [22:08:16] (03CR) 10Eileen: [C: 03+2] CiviCRM Submodule udpate [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824307 (owner: 10Eileen) [22:09:10] dwisehaupt: I want to check for cache misses on prod - do I need to avoid using the redis password on the command line? is it logged? [22:09:54] eileen___: anything on the commandline is kindof easy for other users to see [22:09:59] and then also goes into the bash history [22:10:04] actually I can't run it on redis anyway [22:10:12] -bash: redis-cl: command not found [22:10:15] but you can avoid that last bit by putting spaces before the command [22:10:20] can't run on prod [22:10:52] redis-cli works for me on prod [22:10:56] I think I'm gonna CI the submodule merge myself - [22:11:03] oh - redis-cl [22:11:12] (03CR) 10Eileen: [V: 03+2 C: 03+2] CiviCRM Submodule udpate [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824307 (owner: 10Eileen) [22:11:42] oh, i'll abandon mine, one sec [22:11:49] the patch passed CI so I think the module will too [22:12:02] (03Abandoned) 10Ejegg: Update civicrm submodule [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824304 (owner: 10Ejegg) [22:12:10] (03Abandoned) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/824305 (owner: 10Ejegg) [22:12:29] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/824309 [22:12:37] oh right - we both were doing it [22:13:05] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/824309 (owner: 10Eileen) [22:13:43] yeah. i wish redis had a 'prompt for password' argument. [22:14:02] ejegg: so you put up the merge to deployment concurrent with the submodule commit & it sped things up rather than snarled things up [22:14:27] eileen___: yep, gerrit knows how to sequence merges within the same repo [22:14:38] dwisehaupt: ok - I'm just back to processing that - just looking to see any recurring cache misses [22:14:38] it's only for submodule updates that you need to specify depends-On [22:14:47] ah ok [22:15:29] ejegg: ok - I'm ready to push out that second patch [22:15:44] great, let's see how it goes [22:16:41] !log civicrm upgraded from 4be0724d to 97638e58 [22:16:43] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:19:54] I don't know for sure which line this will likely show on (it wasn't a huge bump like the other on staging anyway) [22:23:30] well if anything that last patch slowed it - but within the margin of error [22:24:18] also based on Redis there could be some other user activity as prevnext cache being hammered [22:28:37] Redis cli doc - you can can avoid password but need to remember the `auth` command - https://wikitech.wikimedia.org/wiki/Fundraising#Performance_tracking [22:31:36] ejegg: I'm pretty sure our speed is slowing down now due to user activitiy - the cache activity is all around [22:31:36] "ZREM" "crm/prevnext/civicrm search CRMContactControllerSearch60rihnrhjio840g [22:31:53] which I think makes the stats for that last patch a bit meaningless :-( [22:32:05] let's see what the processlog looks like [22:32:29] feedback from Australian Green party on the `has` patch https://github.com/civicrm/civicrm-core/pull/24156#issuecomment-1218543735 [22:32:45] nice eileen___ ! [22:32:58] ok, yeah, that search is in the processlog as well [22:32:58] yeah - a bit of validation [22:33:07] but no merges happening at least [22:33:48] yeah - I'm no longer seeing any recurrent cache misses but I think the search is too much noise to know if the patch helped on prod [22:34:06] I think it was only about ten percent at most on staging [22:35:36] dwisehaupt: I wonder if there is any way for us to record heavy user activity in grafana? hammering `crm/prevnext` in Redis is one give away [22:35:51] (that means they are doing searches through the UI) [22:38:47] actually - it is also hit my merge code - I wonder if our scripted dedupe does or only user initiated [22:41:18] dwisehaupt: on prod drush @wmff gives me (errno 13) Permission denied [22:43:38] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Set expiry time for GeoIP cookies - https://phabricator.wikimedia.org/T122097 (10Platonides) [22:48:26] eileen___: hmmm... not sure if we would get that from redis. i can see. [22:49:06] and odd. i just ran a bare `drush @wmff` just fine [22:49:31] `which drush shows` /usr/local/bin/drush [22:49:39] `which drush` that is. [22:52:58] yep - same path [22:53:01] as before, we are seeing a significant number of "crm/prevnext/*/all" queries in the slowlog [22:53:09] Error has occurred executing the Drush script found at ....drush.launcher [22:53:09] (errno 13) Permission denied [22:53:22] oh. interesting. looking. [22:53:40] that prevnext hammering - I guess two theories - its user action or it's the dedupe job [22:54:14] yep the job IS running now [22:54:54] drush.launcher has full execute permissions on the filesystem. [22:55:03] for me? [22:55:15] for everyone. [22:55:55] hmm - that `(errno 13) Permission denied` feels like it means something.... [22:56:06] is there another file being accessed (or not) [22:56:09] yeah. exactly. [22:57:20] I'm starting to suspect `dedupe_civicrm_contacts` not user action for the cache hammer - unfortunately the test doesn't have long enough to go to test much more :-( [22:58:04] so the drush script is essentially a sudo to www-data and then runs the drush command from there. [22:59:21] i just tested as your user and `drush @wmff` runs without error on civi1001 [22:59:34] 10Fundraising Sprint NaN is a Number, 10Fundraising Sprint Overused petting Zoo Memetics, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Investigate queue consumer slowdown in pretest of July 13 2022 - https://phabricator.wikimedia.org/T313000 (10Eileenmcnaughton) Hmm & then... [23:02:30] dwisehaupt: oh well that is weird - I had originally tried from 'where I was' drupal/sites/default [23:02:34] & I get the fail [23:02:47] if I run from my home dir I don't [23:03:16] oh. i bet it tries to write a temp file or something and you don't have write access in that dir. [23:03:27] ah - that makes sense [23:04:17] dedupe job just finished & prevnext caches continue [23:06:43] ejegg: this is weird - https://frmon.frdev.wikimedia.org/d/Pq1YNMviz/fundraising-overview?orgId=1&viewPanel=33&from=now-6h&to=now&refresh=1m - it points to the second patch actually causing slow down - but that doesn't make sense to me - and there is that noise from the user stuff [23:10:31] i need to get the dog out for a quick walk before he makes a mess. back in a bit. [23:17:11] ok - we are inserting option values waay too often in prefixes - could be causing metadata cache flushes [23:24:16] 10Fundraising Sprint NaN is a Number, 10Fundraising Sprint Overused petting Zoo Memetics, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Investigate queue consumer slowdown in pretest of July 13 2022 - https://phabricator.wikimedia.org/T313000 (10Eileenmcnaughton) Hmm Update... [23:28:34] (03PS1) 10Eileen: Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824312 [23:29:32] ejegg: when users enter prefixes is that free-form - or does it only happen in imports? [23:36:20] back [23:38:58] 10fundraising-tech-ops: Fundraising access request for Doris Morgan - https://phabricator.wikimedia.org/T315247 (10Dwisehaupt) [23:39:26] 10fundraising-tech-ops: Fundraising access request for Doris Morgan - https://phabricator.wikimedia.org/T315247 (10Dwisehaupt) 05Open→03Resolved a:03Dwisehaupt Verified civi access in logs. Closing. [23:40:49] (03CR) 10CI reject: [V: 04-1] Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824312 (owner: 10Eileen) [23:44:40] arg dwisehaupt ejegg I screwed up my deploy of the caching fix & reverted in the second deploy - hence the very telling graph - trying again [23:46:18] (03PS1) 10Eileen: CiviCRM submodule update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824313 [23:47:20] (03CR) 10Eileen: [C: 03+2] CiviCRM submodule update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/824313 (owner: 10Eileen) [23:49:47] 10Fundraising Sprint NaN is a Number, 10Fundraising Sprint Overused petting Zoo Memetics, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Investigate queue consumer slowdown in pretest of July 13 2022 - https://phabricator.wikimedia.org/T313000 (10Eileenmcnaughton) UPDATE - my... [23:59:59] 10Fundraising Sprint NaN is a Number, 10Fundraising Sprint Overused petting Zoo Memetics, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Stop prolific doctor breeding - https://phabricator.wikimedia.org/T315509 (10Eileenmcnaughton)