Open
Description
Describe the bug
The ModSecurity log entries show operators and parameters enclosed in backticks (`), while variable names and data values are enclosed in single quotes ('). For example:
- `
PmFromFile' - `
lfi-os-files.data' - `
ARGS:bar' - `
/bin/sh'
and so on...
Wouldn't it have to be?
'PmFromFile''lfi-os-files.data''ARGS:bar''/bin/sh'
Logs and dumps
Output of: AuditLogs
---ABgEnRQg---H--
ModSecurity: Warning. Matched "Operator `PmFromFile' with parameter `lfi-os-files.data' against variable `ARGS:foo' (Value: `/etc/passwd' ) [file "/usr/share/owasp-modsecurity-crs/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf"] [line "99"] [id "930120"] [rev ""] [msg "OS File Access Attempt"] [data "Matched Data: etc/passwd found within ARGS:foo: /etc/passwd"] [severity "2"] [ver "OWASP_CRS/4.14.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language- multi"] [tag "platform-multi"] [tag "attack-lfi"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/ATTACK-LFI"] [tag "capec/1000/255/153/126"] [tag "PCI/6.5.4"] [hostname "192.168.1.35"] [uri "/"] [unique_id "174622258411. 325032"] [ref "o1,10v10,11t:utf8toUnicode,t:urlDecodeUni,t:normalizePathWin"]
ModSecurity: Warning. Matched "Operator `Rx' with parameter `(?i)(?:^|b[\"'\)\[\x5c]*(?:(?:(?:\|\||&&)[\s\x0b]*)?\$[!#\(\*\-0-9\?@_a-\{]*)?\x5c?u[\"'\)\[\x5c]*(?:(?:(?:\|\||&&)[\s\x0b]*)?\$[!#\(\*\-0-9\?@_a-\{]*)?\x5c?s[\"'\)\[\x5c]*(?:(?: (?:\|\||&&)[\s\x0b]*)? (7133 characters omitted)' against variable `ARGS:bar' (Value: `/bin/sh' ) [file "/usr/share/owasp-modsecurity-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "467"] [id "932250"] [rev ""] [msg "Remote Command Execution: Direct Unix Command Execution"] [data "Matched Data: /bin/sh found within ARGS:bar: /bin/sh"] [severity "2"] [ver "OWASP_CRS/4.14.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/ATTACK-RCE"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "192.168.1.35"] [uri "/"] [unique_id "174622258411.325032"] [ref "o0,7v26,7"]
ModSecurity: Warning. Matched "Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:bar' (Value: `/bin/sh' ) [file "/usr/share/owasp-modsecurity-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "606"] [id "932160"] [rev ""] [msg "Remote Command Execution: Unix Shell Code Found"] [data "Matched Data: bin/sh found within ARGS:bar: /bin/sh"] [severity "2"] [ver "OWASP_CRS/4.14.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/ATTACK-RCE"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "192.168.1.35"] [uri "/"] [unique_id "174622258411.325032"] [ref "o1,10v10,11t:cmdLine,t:normalizePatho1,6v26,7t:cmdLine,t:normalizePath"]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `20' ) [file "/usr/share/owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "222"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 20)"] [data ""] [severity "0"] [ver "OWASP_CRS/4.14.0"] [maturity "0"] [accuracy "0"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"] [hostname "192.168.1.35"] [uri "/"] [unique_id "174622258411.325032"] [ref ""]
To Reproduce
- None
Expected behavior
- To know if this behavior is correct.
Server (please complete the following information):
- ModSecurity v3.0.14 with nginx-connector v1.0.3
- WebServer: nginx/1.28.0
- OS: Ubuntu 24.04.2 LTS
Rule Set (please complete the following information):
- OWASP CRS v4.14.0
Additional context
None, just see logs.
Metadata
Metadata
Assignees
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Participants
Activity
airween commentedon May 3, 2025
Hi @wRkA,
thanks for asking this.
I don't know if it's correct or not, but this is the expected behavior :).
If you check the code, the engine produces this message here or here, and the syntax (that you mentioned: the quote marks are
`and'.I'm sure there is a reason why the author of that code has made this decision, but I think it wouldn't be a good idea to replace it now, because many user made their own logparser, and - perhaps - this change would brake them.
airween commentedon Jun 13, 2025
@wRkA,
is there anything that we can do here?
wRkA commentedon Jun 17, 2025
hi @airween
I will try to break down the problem in detail and look for an approach to a real solution. I want to mention that, in the end, these are just ideas; I could be wrong, but we will try to do our best.
Before starting, let's redo a brief summary of the understanding of the log.
Log Entry Structure
A segment begins with a boundary and ends when the next segment begins. The only exception is the terminating segment (Z), which consists only of the boundary. The idea be hind the use of multiple segments is to allow each audit log entry to contain potentially different information. Only parts A and Z are mandatory; the use of the other parts is controlled with the SecAuditLogParts directive. The next contains the list of all audit log parts, along with a description of their purpose.
Example to use: for context
The "problem"
In section
H, it has been found that some parameters appear to start with a backtick (`) and end with a single quote ('), but this raises some questions for us.`) as the opening delimiter and a single quote (') as the closing delimiter for strings in the logs?To try to understand this behavior, we will analyze segment
H, specifically lines1,2, and5, which differ from each other, or at least in the expected message.Line 1
As we can see, the message on line
1does not start with aMatched "Operator, but rather jumps directly to the values related to the rule and the attack type.The detected attack type is
XSS, but why are there no operators here? Is it simply the first warning? In this same line, we already find themsgvalue with the content"XSS Attack Detected via libinjection". As you can see, although the message is almost similar, it can be more specific. Is there anything wrong with this? No, because everything seems to match well, from the messageModSecurity: Warning. detected XSS using libinjection.to the lastrefvalue.In the next line, we will see the inconsistencies.
Line 2
Here comes the interesting part of the log: the line begins with an initial string of
ModSecurity: Warning. Matched "Operator `Rx'. As you can see, theMatched "Operatorat the start of the line represents an inconsistency, or at least it seems so, since the message starts with"Operator `Rx', beginning with"and ending with'. This is where the previous questions arise. Also, the"Operatorstarts with"but does not have a corresponding closing. Is this intentional as well?Line 5
It starts with a message indicating that the response was generated by the server, in this case
403. The pattern repeats: in the"Operator `Ge'it begins with (") and at the end of(Value: `20' )there is no closing ("). Is this behavior normal? Is it intentional? The following values remain consistent with each other.Is it necessary to normalize these inconsistencies? And are they really inconsistencies?
Custom log parsers
So, how does a custom log parser work?
Let's review some points before starting.
Initial structure
We are going to emulate a very basic custom log parser. For this parser, we will rely on simple concepts that will help us normalize log lines with inconsistencies.
Without going into technical details, we will implement the following functions:
With these five functions, we can explore a very basic log parser.
Validations
To validate our log file, in this case "
audit.log", we will perform the following validations:*?|><\"\\`'&;()!~#%+=) that could cause security issues, such as trying to access other folders or execute commands. If the name meets all these rules, it is considered valid; otherwise, it is marked as invalid to protect the system.'A') and end ('Z') of the record, which must have matching identifiers. It ignores very short lines and removes trailing spaces or line breaks. Only if it finds both limits correctly matched does it consider the format valid.'A') and ends with an end mark ('Z'), both with matching identifiers. It also confirms that segments do not overlap, are properly closed at the end of the file, and that at least one valid segment exists. If it finds errors such as missing identifiers, segments out of order, or unclosed segments, it marks the file as poorly structured.Log size & Total count of records in the log
audit.log" file: a start marker ('A') and an end marker ('Z'), which must share the same boundary identifier. Only one valid pair is expected per scan. While scanning the file, it ignores very short lines and trims trailing spaces or newline characters. When it finds the start marker, it saves its identifier and then looks for the matching end marker. Once both are found, it counts the lines between these markers and returns that count along with the total size of the file.Display the last records of the log
Let's get to what really matters: data extraction.
We need to ask ourselves the following questions:
So, based on these simple questions, the data extraction will be as follows.
Segment A:
Segment B:
Segment F:
Example program:
And we
runthe basic command to see the file.Up to this point, we have managed to display a list of the last 20 entries found in the log "
audit.log". What about the other values? The remaining values will be extracted using theBoundary; that is, we will instruct our log parser to identify the Boundary of the file and show us all its details.Detailed analysis of a Boundary
To analyze a specific
Boundaryin theaudit.logfile and guide its use for various purposes (rule modification, creating bans, attack detection, and more...), it is important to identify and highlight the key information it should contain and how to interpret it.Example to extract:
The need to standardize ModSecurity logs for the future largely depends on practical experience in processing and analyzing these records. Creating a specific tool, such as the developed parser, represents a key step to answer this question objectively and with solid grounds.
While some inconsistencies can be handled with robust parsers, normalization could greatly simplify analysis, improve interoperability with other tools, and reduce errors or misunderstandings in incident management.
Fix?
Solution 1:
There is nothing to do.
Solution 2:
Should limiters like backticks (
`) be used at the beginning and end in both cases?Solution 3:
Should single quotes (
') be used at the beginning and end?Solution 4 (For
"):When the message starts with double quotes (
") but does not have them at the end, should it be left as is?I hope this gives us a small premise on how we can approach this "problem."
airween commentedon Jun 17, 2025
Hi @wRkA,
thanks for very detailed answer.
I think the most relevant part is this (and now I can response only for these questions):
Probably yes, but I can't tell you the truth. It's there since the first version was released, for eg see this line.
I think we can say yes, it's intentional.
I have no idea. I say this notation in a few other places (I mean in other projects), so I can't say this is unique.
I don't know, at this time nobody reported this as an issue. And this is why I can't say this is inconsistent. I mean I see what do you mean, if we consider mod_security2 has the "reference" format, then this is different. But if we agree this is intentional, then we can't say this is inconsistent.
Btw I also wrote a parser, it is prepared to recognize the differences (between libmodsecurity3 and mod_security2). (Please note that this parses error.log, which is more-or-less equals with the section
Hfrom audit.log.)Unfortunately I can't say that either. When I wrote libmsclogparser (see above) I reviewed the code lines-by-lines, and handled the differences.
Not yet (I don't remember since I follow (and later maintain/lead) the project).
Because the author of that code created this. I can't provide any better answer, sorry :).
Probably because the
detectXSSis a one-operand operator...? (the one operand is the target; in case of many other operators, you should pass another operand, which will be compared with the target).Sorry, I don't know this either. But here I assume this is by accidentally.
After 7 years, without any reports... what should I say?
Yes, definitely. But now please put yourself in my shoes: after 7 years, while nobody reported any issue about this "problem", and presumably everyone wrote their own parser, which fits all requirements (for themselves), now we change the parser's behavior... Does it seems like a logical step?
But from the other hand: you are right, this seems a bit inconsistent. So we can start to discuss about this, but honestly: my experience is that there will be a very few comments, and that won't help to decide what's the next step.