chore: update Windows build to use PCRE2 as default#3491
Conversation
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
There was a problem hiding this comment.
Pull request overview
Updates the Windows/IIS build tooling and documentation to prefer PCRE2 (and newer dependency versions), aligning with PCRE deprecation and addressing stability concerns like the reported pcre.dll stack overflow.
Changes:
- Bump Windows dependency versions (CMake, PCRE2, zlib, libxml2, Lua, cURL, Apache, ssdeep) and update download URLs.
- Switch Windows build/release scripts and NMake linkage from
pcre.dll/pcre.libtopcre2-8.dll/pcre2-8.lib, enabling PCRE JIT flags in NMake builds. - Refresh
README_WINDOWS.mdbuild instructions to reference PCRE2 and newer dependency versions; CI attempts to enable PCRE JIT.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| iis/download_files.bat | Updates dependency versions and download URLs (including PCRE2). |
| iis/dependencies/build_pcre.bat | Switches dependency build from PCRE to PCRE2 and copies PCRE2 artifacts. |
| iis/build_release.bat | Packages pcre2-8.dll into release artifacts instead of pcre.dll. |
| iis/build_dependencies.bat | Updates dependency version variables used by the batch build pipeline. |
| iis/Makefile.win | Links against pcre2-8.lib and defines WITH_PCRE_JIT. |
| README_WINDOWS.md | Updates Windows build documentation to use PCRE2 and newer dependency versions. |
| .github/workflows/test-ci-windows.yml | Adds -DWITH_PCRE_JIT=ON to the IIS CMake configure step. |
Comments suppressed due to low confidence (2)
README_WINDOWS.md:57
- The updated doc switches the Apache install dir to
C:\Apache2466, but the example still references Apache/httpd 2.4.27 (httpd-2.4.27andHTTPD_BUILD=C:\work\httpd-2.4.27). This is inconsistent and will mislead users following the updated instructions; update these example paths/versions to match the new Apache 2.4.66 guidance.
The directory where you build software from source ( ``C:\work`` in this exmaple)
must contain the Apache source you used to build the Apache web serverand the mod_security source
Apache source is in C:\work\httpd-2.4.27 in this example.
Apache has been installed to C:\Apache2466 in this example.
Mod_security source is in C:\work\mod_security in this example.
## Download and untar the prerequisite library sources:
Download pcre2-10.47.zip from https://github.com/PCRE2Project/pcre2/releases/download/pcre2-10.47/
unzip it into C:\work\ creating C:\work\pcre2-10.47
Download libxml2-2.15.1.tar.gz from https://download.gnome.org/sources/libxml2/2.15/
untar it into C:\work\ creating C:\work\libxml2-2.15.1
Download lua-5.4.8.tar.gz from http://www.lua.org/ftp/
untar it into C:\work\ creating C:\work\lua-5.4.8
Download curl-8.18.0.zip from https://curl.se/download/
unzip it into C:\work\ creating C:\work\curl-8.18.0
Download zlib-1.3.1.tar.gz from https://github.com/madler/zlib/releases/download/v1.3.1/
untar it into C:\work\ creating C:\work\zlib-1.3.1
## Setup your build environment:
1. The ``PATH`` environment variable must include the Visual Studio variables as set by ``vsvars32.bat``
2. The ``PATH`` environment variable must also include the CMAKE ``bin\`` directory
3. Set an environment variable to the Apache source code directory:
SET HTTPD_BUILD=C:\work\httpd-2.4.27
README_WINDOWS.md:204
- The mlogc configuration examples still reference
Apache2427paths (e.g.,CollectorRootandSecAuditLog |C:/Apache2427/...) even though the PR updates the guide toApache2466. These should be updated for consistency, otherwise users will end up with incorrect paths in their configs.
CollectorRoot "C:/Apache2427/logs"
ConsoleURI "https://localhost:8888/rpc/auditLogReceiver"
SensorUsername "test"
SensorPassword "testtest"
LogStorageDir "data"
TransactionLog "mlogc-transaction.log"
QueuePath "mlogc-queue.log"
ErrorLog "mlogc-error.log"
LockFile "mlogc.lck"
KeepEntries 0
ErrorLogLevel 2
MaxConnections 10
MaxWorkerRequests 1000
TransactionDelay 50
StartupDelay 5000
CheckpointInterval 15
ServerErrorTimeout 60
Change the SecAuditLog directive in ``conf\httpd.conf`` to pipe the log data to mlogc instead of writing them to a file:
SecAuditLog |C:/Apache2427/bin/mlogc.bat
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
f4fcb69 to
6334d28
Compare
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 16 out of 20 changed files in this pull request and generated 10 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
@copilot open a new pull request to apply changes based on the open comments in this thread |
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
|
Documentation on Windows should be updated also on the wiki, if this one gets merged (follow up). |
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 21 changed files in this pull request and generated 6 comments.
Comments suppressed due to low confidence (1)
iis/wix/list_dependencies.bat:83
- Several writes to the log file and the final
explorerinvocation use%log_file%without quotes (e.g.,echo. >> %log_file%,>> %log_file%,explorer %log_file%). Since%TEMP%commonly contains spaces (user profile paths), these commands can break or write to the wrong file. Quote%log_file%consistently everywhere it’s used for redirection or as a command argument.
echo. >> %log_file%
REM Updated paths to include Visual Studio 2019 and 2022
set POSSIBLE_PATHS_X86="C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\dumpbin.exe"
set POSSIBLE_PATHS_X64="C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\x86_amd64\dumpbin.exe"
for %%i in (%POSSIBLE_PATHS_X86%) do (
echo Checking for dumpbin x86... %%i
echo Checking for dumpbin x86... %%i >> %log_file%
if exist %%i (
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Fix "installion" typo in Makefile.win usage comments - Fix "mlocg.exe" typo in README_WINDOWS.md - Fix SecAuditLogRelevantStatus regex to use negative lookahead - Check PCRE2 archive existence in build_pcre.bat instead of Apache - Wire up WITH_PCRE_JIT option in iis/CMakeLists.txt - Quote %log_file% paths in list_dependencies.bat for spaces - Use explicit PowerShell path via [SystemFolder] in installer.wxs - Change CreateModSecurityDirs to Return="check" to surface errors - Restore Action="createAndRemoveOnUninstall" on RegistryKey - Only print verbose MSI log on installation failure in CI
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 22 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| C:\work\lua-5.4.8\src\lua54.dll C:\Apache2466\bin\ | ||
| C:\work\libxml2-2.15.1\win32\bin.msvc\libxml2.dll C:\Apache2466\bin\ | ||
| C:\work\curl-8.18.0\builds\libcurl-vc-x64-release-dll-zlib-dll-ipv6-sspi-schannel-obj-lib\libcurl.dll C:\Apache2466\bin\ | ||
| C:\work\mod_security\apache2\mlogc-src\mlogc.exe C:\Apache2466\bin\ |
There was a problem hiding this comment.
The documentation references C:\work\mod_security\apache2\mlogc-src\mlogc.exe, but this repository’s mlogc project lives under the top-level mlogc/ directory (and there is no apache2\mlogc-src folder). Please update this path to the actual built mlogc.exe location so Windows users can follow the instructions successfully.
| C:\work\mod_security\apache2\mlogc-src\mlogc.exe C:\Apache2466\bin\ | |
| C:\work\mod_security\mlogc\mlogc.exe C:\Apache2466\bin\ |
There was a problem hiding this comment.
I think Copilot is right here, there is no such directory. I can't check this now, but could you confirm that Windows build creates apache2\mlogc-src\mlogc.exe, or commit suggestion?
There was a problem hiding this comment.
@airween pushed a39629e4 addressing the three still-open threads:
- README mlogc paths (this thread) — fixed,
mlogc/is the actual top-level directory, updated all three references (build/install copy step, Makefile.win edit step, and themod_security_trunk→mod_securityinconsistency in the build step). iis/dependencies/howto.txtversion list — synced with whatbuild_dependencies.bat/download_files.batactually pin.apache2/modsecurity.hpragma scope — left as-is per your comment, no change.
Also went back through the earlier Feb 8 round where I'd asked @copilot to open follow-up PRs (dumpbin System32/SysWOW64 swap, x86 registry view, PCRE2 lib naming, installer.wxs SysWOW64 block) — those never got a follow-up PR, but checking the current branch tip directly, all four are already fixed from earlier commits, so nothing left there.
One more thing I noticed while checking installer.wxs, unrelated to the above: the InstallModule32/InstallModule64 CustomAction IDs have their content swapped — the action named InstallModule32 actually installs the 64-bit module (System32, bitness64 precondition) and InstallModule64 installs the 32-bit one. Not a functional bug since both run unconditionally and are gated by appcmd's own /preCondition bitness check rather than the ID, but it's confusing to maintain. Leaving it out of this push since it's outside what this thread asked for — happy to open a separate cleanup for it if useful.
Should be ready for another look.
| /* These functions are safe in ModSecurity's usage and required for portability */ | ||
| #pragma warning(disable: 4996) |
There was a problem hiding this comment.
#pragma warning(disable: 4996) in a public header affects every translation unit that includes it, which can unintentionally suppress CRT warnings outside ModSecurity code. Prefer limiting the scope with #pragma warning(push)/pop, or moving this suppression into the Windows build flags (e.g., /wd4996) for the specific targets that need it.
| /* These functions are safe in ModSecurity's usage and required for portability */ | |
| #pragma warning(disable: 4996) | |
| /* These functions are safe in ModSecurity's usage and required for portability */ | |
| #pragma warning(push) | |
| #pragma warning(disable: 4996) | |
| #pragma warning(pop) |
There was a problem hiding this comment.
I think we should ignore this Copilot suggestion now.
| The latest versions of ModSecurity dependencies known to work well are the following: | ||
|
|
||
| cmake-3.8.2-win32-x86 | ||
| pcre-8.40 (patch required and included on file "patch-pcre-8.40.vbs") | ||
| zlib-1.2.11 | ||
| libxml2-2.9.4 | ||
| lua-5.3.4 | ||
| curl-7.54.1 | ||
| httpd-2.4.27 (bin-VC11) | ||
| cmake-3.8.2-win32-x86 (or later) | ||
| pcre2-10.x (PCRE2 is now the default, PCRE 8.x is deprecated) | ||
| - Note: PCRE2 provides better performance and is actively maintained | ||
| zlib-1.2.11 (or later) | ||
| libxml2-2.9.4 (or later) | ||
| lua-5.3.4 (or later) | ||
| curl-7.54.1 (or later) | ||
| httpd-2.4.27 (bin-VC15 or later for VS2019/2022) |
There was a problem hiding this comment.
This “latest versions … known to work well” list still calls out very old versions (e.g., zlib 1.2.11, libxml2 2.9.4, lua 5.3.4, curl 7.54.1, httpd 2.4.27) even though the build scripts in this PR now default to much newer versions. Updating this list to match the versions pinned in iis/build_dependencies.bat/iis/download_files.bat will avoid confusing builders about which versions are actually expected/tested.
There was a problem hiding this comment.
I think here we should collect the latest versions from all components.
|
|
||
| :: BITSAdmin refuses to download YAJL from GitHub URL | ||
| :: @set YAJL_URL=https://github.com/lloyd/yajl/archive/%YAJL:~-9% | ||
| @set YAJL_URL=http://http.debian.net/debian/pool/main/y/yajl/yajl_2.1.0.orig.tar.gz |
There was a problem hiding this comment.
The YAJL_URL download still uses plain HTTP, which allows a network attacker to tamper with the yajl_2.1.0.orig.tar.gz archive in transit and inject malicious code into the libraries you later build and package. Because this script is intended to fetch build-time dependencies for ModSecurity, a compromised YAJL tarball could result in a backdoored module being compiled and shipped. Switch this URL to an authenticated HTTPS source (or add checksum/signature verification) so the YAJL source integrity is cryptographically verified before use.
|
@fzipi many thanks for this excellent PR, especially for the README for Windows building. I really hope we can merge this as soon as possible and we could create a workflow for releases. I've added some notes above beside Copilot suggestions, please take a look at them. |
could you take a look at them above? |
- Fetch YAJL over HTTPS (deb.debian.org) instead of plain HTTP, avoiding tampering of a build-time dependency in transit - Fix mlogc build/install paths in README_WINDOWS.md: mlogc lives under the top-level mlogc/ directory, not apache2/mlogc-src/, and the working-copy root is mod_security, not mod_security_trunk - Sync the "known to work well" dependency version list in iis/dependencies/howto.txt with the versions actually pinned in iis/build_dependencies.bat and iis/download_files.bat Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
|
In the 64-bit installer (Win64=yes), the CustomAction IDs were swapped relative to what they actually install: "InstallModule32" installed the 64-bit module (System32, bitness64 precondition) and "InstallModule64" installed the 32-bit one (SysWOW64, bitness32 precondition). Not a functional bug: both actions run unconditionally on every install and are correctly gated by appcmd's own /preCondition bitness check rather than by the CustomAction ID, and the InstallExecuteSequence references already matched their (mis-)named actions consistently. But the naming makes the file misleading to maintain, and inconsistent with the already-correctly-named UninstallModule32/64 pair. Rename only, no logic/order change: swap the two CustomAction Ids and update the matching InstallExecuteSequence "Custom Action" references so each ID now matches its actual bitness, in the same relative order as before. Found while reviewing owasp-modsecurity#3491. Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>



what
ssdeep 2.14.1)
why
references