MINIFICPP-2668 Move standard processor tests to modular docker tests #2061

lordgamez · 2025-11-10T08:42:59Z

https://issues.apache.org/jira/browse/MINIFICPP-2668

Depends on #2059

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?

For code changes:

If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE file?
If applicable, have you updated the NOTICE file?

For documentation related changes:

Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

Copilot

Pull Request Overview

This PR migrates standard processor tests from the legacy docker integration test framework to the newer modular behave-based test framework. The changes consolidate testing infrastructure by removing duplicate processor implementations and container classes from the old framework.

Key changes:

Adds new container implementations (TcpClientContainer, DiagSlave) to the modular test framework
Updates 8 feature files to use the new framework's syntax and step definitions
Removes deprecated processor and container classes from the old docker integration framework
Extends the behave framework with new step definitions and helper methods

Reviewed Changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tcp_client_container.py	New TCP client container for network listener tests
diag_slave_container.py	New DiagSlave container for Modbus testing
steps.py	Adds step definitions for PLC/Modbus and TCP client setup
*.feature files (8 files)	Migrated to new framework syntax with updated assertions
environment.py	Adds platform detection for x86/x64-only tests
core_steps.py	New steps for file creation and FIPS mode
checking_steps.py	New assertion steps for JSON and empty file validation
container.py	Enhanced with JSON validation and debug print statement
minifi_container.py	Adds FIPS mode support
Old framework files	Removes deprecated processor/container implementations

Comments suppressed due to low confidence (1)

extensions/standard-processors/tests/features/defragtextflowfiles.feature:42

The expected output appears to have changed from having one '<1>,' to two '<1>,<1>,' at the beginning. This change affects multiple test scenarios (lines 42 and 69). Ensure this change reflects the actual expected behavior and that corresponding tests verify this is correct, as this could indicate a change in how the DefragmentText processor handles end-of-message patterns.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

behave_framework/src/minifi_test_framework/containers/container.py

extensions/standard-processors/tests/features/steps/tcp_client_container.py

extensions/standard-processors/tests/features/steps/diag_slave_container.py

behave_framework/src/minifi_test_framework/containers/container.py

szaszm · 2025-12-03T17:31:48Z

extensions/standard-processors/tests/features/defragtextflowfiles.feature

      | input_one                                    | input_two                                        | pattern       | pattern location | success_flow_files                                                          |
      | <1>cat%dog%mouse%<2>apple%banana%<3>English% | <1>Katze%Hund%Maus%<2>Apfel%Banane%<3>Deutsch%   | <[0-9]+>      | Start of Message | <1>cat%dog%mouse%,<1>Katze%Hund%Maus%,<2>apple%banana%,<2>Apfel%Banane%       |
-      | <1>cat%dog%mouse%<2>apple%banana%<3>English% | <1>Katze%Hund%Maus%<2>Apfel%Banane%<3>Deutsch%   | <[0-9]+>      | End of Message   | <1>,cat%dog%mouse%<2>,Katze%Hund%Maus%<2>,apple%banana%<3>,Apfel%Banane%<3>   |
+      | <1>cat%dog%mouse%<2>apple%banana%<3>English% | <1>Katze%Hund%Maus%<2>Apfel%Banane%<3>Deutsch%   | <[0-9]+>      | End of Message   | <1>,<1>,cat%dog%mouse%<2>,Katze%Hund%Maus%<2>,apple%banana%<3>,Apfel%Banane%<3>   |


What changed here?

szaszm · 2025-12-03T18:25:23Z

extensions/standard-processors/tests/features/steps/tcp_client_container.py

+        cmd = (
+            "/bin/sh -c 'apk add netcat-openbsd && "
+            "echo TCP client container started; "
+            "while true; do echo test_tcp_message | "
+            f"nc minifi-primary-{test_context.scenario_id} 10254; "
+            "sleep 1; done'"
+        )


is wrapping in sh -c necessary? I suspect it's not, because the command is given as a single string, meaning it has to go through a command line interpreter before being executed.

Suggested change

cmd = (

"/bin/sh -c 'apk add netcat-openbsd && "

"echo TCP client container started; "

"while true; do echo test_tcp_message | "

f"nc minifi-primary-{test_context.scenario_id} 10254; "

"sleep 1; done'"

)

cmd = (

"apk add netcat-openbsd && "

"echo TCP client container started; "

"while true; do"

f" echo test_tcp_message | nc minifi-primary-'{test_context.scenario_id}' 10254; "

" sleep 1;"

"done"

)

szaszm · 2025-12-03T18:31:28Z

extensions/standard-processors/tests/features/evaluate_json_path.feature

-    And the Minifi logs contain the following message: "key:author value:null" in less than 0 seconds
-    And the Minifi logs do not contain the following message: "key:release" after 0 seconds
+    And the Minifi logs contain the following message: "key:author value:null" in less than 1 seconds
+    And the Minifi logs do not contain the following message: "key:release" after 10 seconds


why were the timeouts changed?

I can only speak for myself, but I've encountered a bug in the framework where if yhe timeout is too low it wont even check it once, that should be fixed then I think these can be reverted

I find the 10sec one a big high after another 10sec wait above these lines, but I have no problem with them if they're necessary.

fgerlits · 2025-12-05T13:31:36Z

behave_framework/src/minifi_test_framework/containers/container.py

+        return actual_file_contents
+
+    def _verify_file_contents_in_running_container(self, directory_path: str, expected_contents: list[str]) -> bool:
+        if not self.not_empty_dir_exists(directory_path):


nitpicking, and belongs to a previous PR, but the name of this function should be

Suggested change

if not self.not_empty_dir_exists(directory_path):

if not self.nonempty_dir_exists(directory_path):

fgerlits · 2025-12-05T15:57:29Z

extensions/standard-processors/tests/features/steps/steps.py

    context.containers["syslog-udp"] = SyslogContainer("udp", context)
+
+
+@step(u'there is an accessible PLC with modbus enabled')


the u is a leftover from Python 2, I think, and can be removed

fgerlits · 2025-12-05T16:05:22Z

extensions/standard-processors/tests/features/file_system_operations.feature

      | processor name | property name   | property value |
      | GetFile        | Input Directory | /tmp/input     |
-      | PutFile_1      | Input Directory | /tmp           |
+      | PutFile_1      | Directory       | /tmp           |


interesting -- how was this working before? PutFile has no Input Directory property

lordgamez force-pushed the MINIFICPP-2668 branch from 5e51067 to 1b11558 Compare November 10, 2025 09:26

lordgamez added depends-on-another-PR low-impact Test only or trivial change that's most likely not gonna introduce any new bugs labels Nov 11, 2025

lordgamez mentioned this pull request Nov 12, 2025

MINIFICPP-2674 Move Couchbase tests to modular docker tests #2066

Open

8 tasks

lordgamez force-pushed the MINIFICPP-2668 branch 2 times, most recently from b6a74cc to 67bf898 Compare November 13, 2025 13:17

fgerlits removed the depends-on-another-PR label Nov 14, 2025

lordgamez force-pushed the MINIFICPP-2666 branch from e172fa7 to 2640d9b Compare November 17, 2025 10:24

lordgamez force-pushed the MINIFICPP-2668 branch from 67bf898 to d86d34a Compare November 17, 2025 10:24

lordgamez added the depends-on-another-PR label Nov 17, 2025

lordgamez requested a review from Copilot November 17, 2025 12:23

Copilot started reviewing on behalf of lordgamez November 17, 2025 12:24 View session

Copilot finished reviewing on behalf of lordgamez November 17, 2025 12:25

Copilot AI reviewed Nov 17, 2025

View reviewed changes

lordgamez force-pushed the MINIFICPP-2668 branch 4 times, most recently from f4bbe06 to 5a197d8 Compare November 17, 2025 16:11

MINIFICPP-2668 Move standard processor tests to modular docker tests

667d6bd

lordgamez force-pushed the MINIFICPP-2666 branch from 2640d9b to 985912c Compare November 21, 2025 09:23

lordgamez force-pushed the MINIFICPP-2668 branch from 5a197d8 to 667d6bd Compare November 21, 2025 09:23

szaszm reviewed Dec 3, 2025

View reviewed changes

fgerlits reviewed Dec 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

MINIFICPP-2668 Move standard processor tests to modular docker tests #2061

MINIFICPP-2668 Move standard processor tests to modular docker tests #2061

Uh oh!

lordgamez commented Nov 10, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

szaszm Dec 3, 2025

Uh oh!

szaszm Dec 3, 2025

Uh oh!

szaszm Dec 3, 2025

Uh oh!

martinzink Dec 4, 2025

Uh oh!

szaszm Dec 4, 2025

Uh oh!

fgerlits Dec 5, 2025

Uh oh!

fgerlits Dec 5, 2025

Uh oh!

fgerlits Dec 5, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	if not self.not_empty_dir_exists(directory_path):
	if not self.nonempty_dir_exists(directory_path):

		context.containers["syslog-udp"] = SyslogContainer("udp", context)


		@step(u'there is an accessible PLC with modbus enabled')

MINIFICPP-2668 Move standard processor tests to modular docker tests #2061

Are you sure you want to change the base?

MINIFICPP-2668 Move standard processor tests to modular docker tests #2061

Uh oh!

Conversation

lordgamez commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

For all changes:

For code changes:

For documentation related changes:

Note:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

szaszm Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

szaszm Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

szaszm Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

martinzink Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

szaszm Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

fgerlits Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

fgerlits Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

fgerlits Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

lordgamez commented Nov 10, 2025 •

edited

Loading

fgerlits Dec 5, 2025 •

edited

Loading