Only record the matching pattern from a traffic shaping rule to log on internal transient failure

[lianatech-jutaky]

What Operating System are you seeing this problem on?

Debian 12

What Hardware is this system running?

Cloud instance

KumoMTA version

2025.02.28.084144.3966ca59

Did you try the latest release to see if the issue is better (or worse!) than your current version?

Yes, and I updated the version box above to show the version that I tried

Describe the bug

Scenario: traffic automation helper toml has one gigantic rule with dozens of regex patterns to match various responses. Logically similar patterns, so they all trigger one identical config change.

Behavior: when internal transient failure occurs due to a queue suspension, KumoMTA implodes the regex pattern list into one single mile long string into the log:

451: KumoMTA internal: scheduled queue is suspended: automation rule: $very_long_text_with_all_the_regex_patterns_imploded

When lots of internal transient failures happen, this will suck up disk space and IO quite a bit into logging efforts.

My suggestion is to only log the relevant regex pattern that actually matched and triggered the suspension. No reason to include all the dozens of other unrelated patterns in there.

This would reduce resource usage as well as improve UX by actually showing in the log what triggered the suspension.

To Reproduce

No response

Configuration

Transient failure logging enabled and a traffic shaping rule with maany regexes.

Expected Behavior

No response

Anything else?

No response

[lianatech-jutaky]

Out of curiosity I had a look at the code and I came up with the following diff. I didn't test beyond compilation as there are probably all kind of sins as I haven't done Rust before. But this is the idea I would like to see in practice:

--- a/crates/kumo-api-types/src/shaping.rs
+++ b/crates/kumo-api-types/src/shaping.rs
@@ -250,6 +250,16 @@ impl Rule {
             .any(|r| r.is_match(response).unwrap_or(false))
     }
 
+    pub fn get_matching_regex(&self, response: &str) -> Option<&Regex> {
+        for re in self.regex.iter() {
+            if re.is_match(response).unwrap() {
+                return Some(re);
+            }
+        }
+
+        None
+    }
+
     pub fn clone_and_set_rollup(&self) -> Self {
         let mut result = self.clone();
         result.was_rollup = true;
diff --git a/crates/tsa-daemon/src/http_server.rs b/crates/tsa-daemon/src/http_server.rs
index f250a60d..5c5b4ce3 100644
--- a/crates/tsa-daemon/src/http_server.rs
+++ b/crates/tsa-daemon/src/http_server.rs
@@ -319,7 +319,7 @@ fn create_tenant_suspension(
 
     let mut reason = format!(
         "automation rule: {} tenant={tenant} domain={}",
-        regex_list_to_string(&rule.regex),
+        rule.get_matching_regex(&record.response.content).unwrap().to_string(),
         components.domain
     );
     if let Some(campaign) = &campaign {

[wez]

The code change you're proposing raises the cost of matching rules by increasing the big-O factor. In a busy system, that is typically undesirable.

Can you share concrete examples of the rules that are causing issues? I haven't seen anything large enough to warrant this sort of effort from our other users so far.

[lianatech-jutaky]

We have rules with ten or so regexes and the text strings in the patterns can be long. The core of the problem is that internal TransientFailure log lines are hard to read at the moment and deliverability engineers have hard time deciphering why the email truly experienced a failure. One can't search using the email id, because that email was not the one triggering the suspension and the log line itself lists all the possible causes for the suspension.

It becomes a needle in a haystack search then to relocate the original text that triggered the rule originally. Usually it is done based on guesses on timestamps and sender address. Very inefficient use of human resources when code could just write the matching pattern into reason field which is logged with the fails.

That code change is in the suspension path - do even big systems really hit that path that often?

I guess as a workaround, we will split our traffic automation rules to not use regex lists and only have one regex per rule and transfer the overhead to rule duplication and management.