Operator @fuzzyHash don't work as expected

[capy3ra]

Describe the bug

I had enabled SecUploadKeepFiles On.
Run this ssdeep command to create hash of test.txt file
ssdeep -b /opt/modsecurity/var/test.txt > /opt/modsecurity/var/test.hash
Rule content:
SecRule FILES_TMP_CONTENT "@fuzzyHash /opt/modsecurity/var/test.hash 10"
Then upload a file with the same content as test.txt.
But it does not work.

Logs and dumps

Output of:

1. DebugLogs (level 9):

[176250032761.172639] [] [4] Initializing transaction
[176250032761.172639] [] [4] Transaction context created.
[176250032761.172639] [] [4] Starting phase CONNECTION. (SecRules 0)
[176250032761.172639] [] [9] This phase consists of 0 rule(s).
[176250032761.172639] [] [4] Starting phase URI. (SecRules 0 + 1/2)
[176250032761.172639] [/Default.aspx] [4] Starting phase REQUEST_HEADERS.  (SecRules 1)
[176250032761.172639] [/Default.aspx] [9] This phase consists of 2 rule(s).
[176250032761.172639] [/Default.aspx] [4] (Rule: 200000) Executing operator "Rx" with param "^(?:application(?:/soap\+|/)|text/)xml" against REQUEST_HEADERS:Content-Type.
[176250032761.172639] [/Default.aspx] [9]  T (0) t:lowercase: "multipart/form-data; boundary=----geckoformboundaryf235087e71d4ac8acabb7bef6d858 (3 characters omitted)"
[176250032761.172639] [/Default.aspx] [9] Target value: "multipart/form-data; boundary=----geckoformboundaryf235087e71d4ac8acabb7bef6d858 (3 characters omitted)" (Variable: REQUEST_HEADERS:Content-Type)
[176250032761.172639] [/Default.aspx] [4] Rule returned 0.
[176250032761.172639] [/Default.aspx] [9] Matched vars cleaned.
[176250032761.172639] [/Default.aspx] [4] (Rule: 200001) Executing operator "Rx" with param "^application/json" against REQUEST_HEADERS:Content-Type.
[176250032761.172639] [/Default.aspx] [9]  T (0) t:lowercase: "multipart/form-data; boundary=----geckoformboundaryf235087e71d4ac8acabb7bef6d858 (3 characters omitted)"
[176250032761.172639] [/Default.aspx] [9] Target value: "multipart/form-data; boundary=----geckoformboundaryf235087e71d4ac8acabb7bef6d858 (3 characters omitted)" (Variable: REQUEST_HEADERS:Content-Type)
[176250032761.172639] [/Default.aspx] [4] Rule returned 0.
[176250032761.172639] [/Default.aspx] [9] Matched vars cleaned.
[176250032761.172639] [/Default.aspx] [9] Adding request body: 16981 bytes. Limit set to: -939524096.000000
[176250032761.172639] [/Default.aspx] [9] Appending request body: 16981 bytes. Limit set to: -939524096.000000
[176250032761.172639] [/Default.aspx] [4] Starting phase REQUEST_BODY. (SecRules 2)
[176250032761.172639] [/Default.aspx] [9] Multipart: Boundary: ----geckoformboundaryf235087e71d4ac8acabb7bef6d858c5b
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header "Content-Disposition" "form-data; name="__VIEWSTATE"".
[176250032761.172639] [/Default.aspx] [9] Multipart: Content-Disposition name: __VIEWSTATE.
[176250032761.172639] [/Default.aspx] [9] Multipart: Added data to variable: /wEPDwUJMjIyMjQwMzc3D2QWAgIBD2QWAgIFDxYCHgRUZXh0BSlTYXZlZDogfi91cGxvYWRzL3JhaW5fd2ViYXR0YWNrX2Z0X3hnYi5weWRkmPL1vpP+8jMwHzq/v58z8qKsctI2CH9myxwguxttGus=
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header line:Content-Disposition: form-data; name="__VIEWSTATE"
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part to the list: name "__VIEWSTATE" (offset 111, length 152)
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header "Content-Disposition" "form-data; name="__VIEWSTATEGENERATOR"".
[176250032761.172639] [/Default.aspx] [9] Multipart: Content-Disposition name: __VIEWSTATEGENERATOR.
[176250032761.172639] [/Default.aspx] [9] Multipart: Added data to variable: CA0B0334
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header line:Content-Disposition: form-data; name="__VIEWSTATEGENERATOR"
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part to the list: name "__VIEWSTATEGENERATOR" (offset 385, length 8)
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header "Content-Disposition" "form-data; name="__EVENTVALIDATION"".
[176250032761.172639] [/Default.aspx] [9] Multipart: Content-Disposition name: __EVENTVALIDATION.
[176250032761.172639] [/Default.aspx] [9] Multipart: Added data to variable: /wEdAALkRK27CPbw/jnsaM8B1/5qzfg78Z8BXhXifTCAVkevd/xzBmfSPTKddd/pf0mSyYWDYAiPyvXWDU+d74+Qz5RS
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header line:Content-Disposition: form-data; name="__EVENTVALIDATION"
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part to the list: name "__EVENTVALIDATION" (offset 512, length 92)
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header "Content-Disposition" "form-data; name="FileUpload1"; filename="rain_webattack_ft_xgb.py"".
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header "Content-Type" "application/octet-stream".
[176250032761.172639] [/Default.aspx] [9] Multipart: Content-Disposition name: FileUpload1.
[176250032761.172639] [/Default.aspx] [9] Multipart: Content-Disposition filename: rain_webattack_ft_xgb.py.
[176250032761.172639] [/Default.aspx] [4] MultipartPartTmpFile: Create filename= /opt/modsecurity/var/upload/20251107-142527-176250032761.172639-file-X0Gmez
[176250032761.172639] [/Default.aspx] [4] Multipart: Created temporary file 1 (mode o511): /opt/modsecurity/var/upload/20251107-142527-176250032761.172639-file-X0Gmez
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header line:Content-Disposition: form-data; name="FileUpload1"; filename="rain_webattack_ft_xgb.py"
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header line:Content-Type: application/octet-stream
[176250032761.172639] [/Default.aspx] [9] Multipart: Added file part to the list: name "FileUpload1" file name "rain_webattack_ft_xgb.py" (offset 794, length 16011)
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header "Content-Disposition" "form-data; name="Button1"".
[176250032761.172639] [/Default.aspx] [9] Multipart: Content-Disposition name: Button1.
[176250032761.172639] [/Default.aspx] [9] Multipart: Added data to variable: Upload
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part header line:Content-Disposition: form-data; name="Button1"
[176250032761.172639] [/Default.aspx] [9] Multipart: Added part to the list: name "Button1" (offset 16914, length 6)
[176250032761.172639] [/Default.aspx] [4] Adding request argument (BODY): name "__VIEWSTATE", value "/wEPDwUJMjIyMjQwMzc3D2QWAgIBD2QWAgIFDxYCHgRUZXh0BSlTYXZlZDogfi91cGxvYWRzL3JhaW5fd2ViYXR0YWNrX2Z0X3hnYi5weWRkmPL1vpP+8jMwHzq/v58z8qKsctI2CH9myxwguxttGus="
[176250032761.172639] [/Default.aspx] [4] Adding request argument (BODY): name "__VIEWSTATEGENERATOR", value "CA0B0334"
[176250032761.172639] [/Default.aspx] [4] Adding request argument (BODY): name "__EVENTVALIDATION", value "/wEdAALkRK27CPbw/jnsaM8B1/5qzfg78Z8BXhXifTCAVkevd/xzBmfSPTKddd/pf0mSyYWDYAiPyvXWDU+d74+Qz5RS"
[176250032761.172639] [/Default.aspx] [4] Adding request argument (BODY): name "Button1", value "Upload"
[176250032761.172639] [/Default.aspx] [4] Multipart: Cleanup started (keep files set to True)
[176250032761.172639] [/Default.aspx] [9] This phase consists of 6 rule(s).
[176250032761.172639] [/Default.aspx] [4] (Rule: 200007) Executing operator "Ge" with param "1000" against ARGS.
[176250032761.172639] [/Default.aspx] [9] Target value: "4" (Variable: ARGS)
[176250032761.172639] [/Default.aspx] [4] Rule returned 0.
[176250032761.172639] [/Default.aspx] [9] Matched vars cleaned.
[176250032761.172639] [/Default.aspx] [4] (Rule: 200002) Executing operator "Eq" with param "0" against REQBODY_ERROR.
[176250032761.172639] [/Default.aspx] [9] Target value: "0" (Variable: REQBODY_ERROR)
[176250032761.172639] [/Default.aspx] [4] Rule returned 0.
[176250032761.172639] [/Default.aspx] [9] Matched vars cleaned.
[176250032761.172639] [/Default.aspx] [4] (Rule: 200003) Executing operator "Eq" with param "0" against MULTIPART_STRICT_ERROR.
[176250032761.172639] [/Default.aspx] [9] Target value: "0" (Variable: MULTIPART_STRICT_ERROR)
[176250032761.172639] [/Default.aspx] [4] Rule returned 0.
[176250032761.172639] [/Default.aspx] [9] Matched vars cleaned.
[176250032761.172639] [/Default.aspx] [4] (Rule: 200004) Executing operator "Eq" with param "1" against MULTIPART_UNMATCHED_BOUNDARY.
[176250032761.172639] [/Default.aspx] [9] Target value: "2" (Variable: MULTIPART_UNMATCHED_BOUNDARY)
[176250032761.172639] [/Default.aspx] [4] Rule returned 0.
[176250032761.172639] [/Default.aspx] [9] Matched vars cleaned.
[176250032761.172639] [/Default.aspx] [4] (Rule: 200005) Executing operator "StrEq" with param "0" against TX:regex(^MSC_).
[176250032761.172639] [/Default.aspx] [4] Rule returned 0.
[176250032761.172639] [/Default.aspx] [9] Matched vars cleaned.
[176250032761.172639] [/Default.aspx] [4] (Rule: 20483661231) Executing operator "FuzzyHash" with param "/opt/modsecurity/var/test.hash 10" against FILES_TMP_CONTENT.
[176250032761.172639] [/Default.aspx] [9] Target value: "#!/usr/bin/env python3"""Nginx WebAttack Detection \xe2\x80\x93 fastText + XGBoo (15110 characters omitted)" (Variable: FILES_TMP_CONTENT:FileUpload1)
[176250032761.172639] [/Default.aspx] [4] Rule returned 0.
[176250032761.172639] [/Default.aspx] [9] Matched vars cleaned.

Expected behavior

Block this request and in debug log indicate match score

Server:

• ModSecurity version: ModSecurity-nginx v1.0.3 libmodsecurity3 version 3.0.12
• WebServer: nginx-1.18.0
• OS: [ubuntu]

[airween]

Hi @capy3ra,

thanks for reporting this - my first idea and request to you that could you check library's @fuzzyHash operator's test?

Here you can see:
https://github.com/owasp-modsecurity/ModSecurity/blob/v3/master/test/test-cases/regression/operator-fuzzyhash.json

Do you think those tests are correct?

[capy3ra]

it work. But i want to inspect file upload by Content-Type: multipart/form-data. is it possible?