[06:53:19] dcausse / zpapierski / addshore: looks like the drop in triple count leveled off and we're now back to the expected growth. seems like it was probably organic? https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&var-cluster_name=wdqs&from=1624351724889&to=1624430473071 [06:56:38] ryankemper: yes I hope, it's also around when we started lagging and might have caused slowdowns on some bots, perhaps bots bulk-adding data were stopped while others doing cleanups continued? (just random guesses) [06:59:00] or we starting lagging because there were more deletions? can't really tell... :/ [07:02:22] not sure if we have a way to see just the deletion events but I suspect we did have more deletion events than normal, because it feels like if we extrapolate from the growth before we started decreasing, that we'd be at a higher level than we ended up being [07:02:45] that's just me eyeballing it though not actually calculating the slope [07:06:22] yes... although it's hard to gauge deletions, it might not only beb actual page deletes but simply removing statements from items which are seen as normal edits [07:10:11] the new updater is supposed to show more granular metrics about that https://grafana-rw.wikimedia.org/d/fdU5Zx-Mk/wdqs-streaming-updater?orgId=1&from=now-2d&to=now but can't make any sense of the triple ingestion rate, would have hoped to see a pattern on the number of deleted triples but no... something might not be captured properly [08:52:12] very surprised to see that language coverage on description is a lot higher than labels (c.f. https://wikitech.wikimedia.org/wiki/User:AKhatun/Wikidata_Vertical_Analysis) [10:04:38] lunch [11:20:21] dcausse: I think it was some merges [11:20:37] and I guess merges (which leave a redirect) don't actually count as deletes? [11:21:29] did the edit pattern cause some issues / fun for the updater? [12:11:41] addshore: thanks, yes, I guess merges are just normal edits and go to revision-create? it did not cause any problem except the lag but unsure how this is all related [12:14:47] break [12:28:08] yes, just revision-create [17:35:26] dinner [21:22:16] ebernhardson: just to make sure I understand, the mjolnir bulk daemon now uses the `--priority-topic` flag right? as of this patch https://gerrit.wikimedia.org/r/c/search/MjoLniR/deploy/+/699808/ [21:41:52] ryankemper: yup, i deployed that and it's ready to go [21:43:01] cool, I merged https://gerrit.wikimedia.org/r/c/operations/puppet/+/699814 to start using that flag [21:47:57] sweet, thanks!