feat(#2374): Add suggestion to test processing elements #2375

tenthe · 2023-12-27T10:19:57Z

Purpose

The current pull request introduces a prototype for unit testing processing elements in the streampipes-processors-filter-jvm module, specifically for the NumericalFilterProcessor.

Proposal

Implemented a basic prototype allowing developers to test processing element configurations.
Used Map<String, Object> for user input and List<Map<String, Object>> for events.
Proposing a discussion on whether to keep the current data type approach for simplicity or consider alternatives.
A static class, ProcessingElementTestExecutor, is suggested to execute the actual tests.

Important

This PR is not final; its purpose is to showcase the testing functionality and initiate a discussion on testing options.

If you have any suggestions or opinions on the proposed approach, feel free to share them here.

Remarks

PR introduces (a) breaking change(s): no

PR introduces (a) deprecation(s): no

github-actions · 2024-01-18T01:28:16Z

Hello there 👋

We noticed that it's been some time since activity occurred on your pull request 🤔. In order to keep things moving forward, we're marking this PR as stale and giving you 7 days to respond before it's automatically closed ⏰.

Please take a moment to review your pull request and make any necessary updates or changes 👨‍💻. If you need more time or have any questions, please don't hesitate to let us know 💬.

Thank you for your contributions to our project, and we look forward to hearing back from you soon 🙏.

github-actions · 2024-01-25T01:29:25Z

Hello there 👋

Unfortunately, we didn't hear back from you regarding your pull request, so we're closing it now. Don't worry, you can always reopen the PR at any time if you wish to continue working on it 🙌.

Please note that the branch associated with this pull request will not be deleted, so you can still access your changes and continue to work on them as needed 💻.

Thank you for your contributions to our project, and we hope to see you again soon!

…processing-elements

IsaakKrut · 2024-04-04T16:28:03Z

@tenthe do you have examples of tests or processors that use selector prefix or event schema? Are there any that use multiple prefixes? Trying to understand which processors require something extra for testing

tenthe · 2024-04-05T05:28:47Z

Hi @IsaakKrut,

my suggestion would be that you try to write a test for another processor with this "new" API.
And then, based on this experience, we discuss and try to design the API and the implementation for the "ProcessingElementTestExecutor" together.
What do you think of this idea?

I think the biggest challenge will be to handle all the different types of static properties that a user can enter to configure the processor. I think one or two more sample tests would help to get a better understanding of what the best solution should look like.

Cheers,
Philipp

…nts (#2730) * #2374 refactored unit tests to use ProcessingElementTestExecutor * #2374 cleanup * #2374 cleanup * #2374 updated test executor to non-static methods * #2374 updated test executor to non-static methods

…ly-unit-test-processing-elements' into 2374-provide-a-framework-to-easily-unit-test-processing-elements # Conflicts: # streampipes-extensions/streampipes-processors-filters-jvm/src/test/java/org/apache/streampipes/processors/filters/jvm/processor/booleanfilter/TestBooleanFilterProcessor.java # streampipes-extensions/streampipes-processors-filters-jvm/src/test/java/org/apache/streampipes/processors/filters/jvm/processor/compose/TestComposeProcessor.java # streampipes-extensions/streampipes-processors-filters-jvm/src/test/java/org/apache/streampipes/processors/filters/jvm/processor/merge/TestMergeByTimeProcessor.java # streampipes-extensions/streampipes-processors-filters-jvm/src/test/java/org/apache/streampipes/processors/filters/jvm/processor/numericalfilter/NumericalFilterProcessorTest.java # streampipes-extensions/streampipes-processors-filters-jvm/src/test/java/org/apache/streampipes/processors/filters/jvm/processor/sdt/TestSwingingDoorTrendingFilterProcessor.java # streampipes-extensions/streampipes-processors-filters-jvm/src/test/java/org/apache/streampipes/processors/filters/jvm/processor/textfilter/TestTextFilterProcessor.java # streampipes-test-utils-executors/src/test/java/org/apache/streampipes/test/executors/ProcessingElementTestExecutor.java

pambrocio · 2024-04-25T08:54:17Z

Hi @tenthe @IsaakKrut,

I would love to contribute to this effort. I have already migrated the classes stated in #2538 to JUnit 5, I am quite unclear on what needs to be done for my work to integrate into this effort.

Thanks,
Pambrocio

tenthe · 2024-04-26T06:49:10Z

Hi @tenthe @IsaakKrut,

I would love to contribute to this effort. I have already migrated the classes stated in #2538 to JUnit 5, I am quite unclear on what needs to be done for my work to integrate into this effort.

Thanks, Pambrocio

Hi @pambrocio,
thanks a lot for your PRs. I do not have much time this week, I'll have a detailed look at them next week.

Cheers,
Philipp

tenthe · 2024-04-29T16:32:19Z

Hi @IsaakKrut,
I have fixed the problem with the cyclic dependencies. I also fixed the checkstyle issues.

Is it possible for you to extract the TestConfigurationBuilder into an own file?

@pambrocio, I suggest that we clean up this branch and then merge it into dev.
Based on these changes, you can customize your PRs as the API should now be cleaner.
And we can then use your feedback to further improve the testing API for processors.
Is that ok for you?

Cheers,
Philipp

bossenti · 2024-04-05T06:37:55Z

...eampipes/processors/filters/jvm/processor/numericalfilter/ProcessingElementTestExecutor.java

+  ) {
+
+    // initialize the extractor with the provided configuration of the user input
+    var e = getProcessingElementParameterExtractor(processor, userConfiguration);


Suggested change

var e = getProcessingElementParameterExtractor(processor, userConfiguration);

var extractor = getProcessingElementParameterExtractor(processor, userConfiguration);

bossenti · 2024-04-05T06:38:45Z

...eampipes/processors/filters/jvm/processor/numericalfilter/ProcessingElementTestExecutor.java

+                                .getRaw()
+             ));
+
+    // vlaidate that the processor is stopped correctly


Suggested change

// vlaidate that the processor is stopped correctly

// validate that the processor is stopped correctly

feat(#2374): Add suggestion to test processing elements

eea898c

tenthe linked an issue Dec 27, 2023 that may be closed by this pull request

Provide a framework to easily unit test processing elements #2374

Closed

github-actions bot added the stale Marks pull requests that are classified as `stale` by our bot. label Jan 18, 2024

github-actions bot closed this Jan 25, 2024

tenthe reopened this Apr 4, 2024

tenthe mentioned this pull request Apr 4, 2024

#2545 migrated TestBooleanCounterProcessor parameterized test to junit5 #2683

Merged

Merge branch 'dev' into 2374-provide-a-framework-to-easily-unit-test-…

8fa927d

…processing-elements

github-actions bot removed the stale Marks pull requests that are classified as `stale` by our bot. label Apr 5, 2024

IsaakKrut added 6 commits April 12, 2024 16:34

#2374 refactored unit tests to use ProcessingElementTestExecutor

d3edbd5

#2374 cleanup

02dab2d

#2374 cleanup

1b33586

#2374 updated test executor to non-static methods

ccfe4ad

#2374 updated test executor to non-static methods

4769f24

bossenti mentioned this pull request Apr 22, 2024

TestSizeMeasureProcessor #2754

Open

tenthe marked this pull request as ready for review April 24, 2024 07:46

IsaakKrut added 3 commits April 24, 2024 15:21

#2374 Added test configuration builder and resolved selector prefixes

c25b1be

#2374 Moved test executor to a new package

d9f268e

bossenti mentioned this pull request Apr 25, 2024

Fix JUnit Parameterized Tests #2538

Open

13 tasks

feat(#2374): Fix cyclic dependencies and checkstyle

5735eee

tenthe and others added 2 commits April 29, 2024 22:34

feat(#2374): Fix checkstyle

263f2dd

#2374 extracted TestConfigurationBuilder into a separate file

4eca0df

bossenti approved these changes Apr 30, 2024

View reviewed changes

tenthe merged commit d69504c into dev May 2, 2024
21 checks passed

tenthe deleted the 2374-provide-a-framework-to-easily-unit-test-processing-elements branch May 2, 2024 06:38

bossenti linked an issue May 2, 2024 that may be closed by this pull request

Unit testing framework updates #2737

Closed

bossenti mentioned this pull request May 2, 2024

Unit testing framework updates #2737

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(#2374): Add suggestion to test processing elements #2375

feat(#2374): Add suggestion to test processing elements #2375

tenthe commented Dec 27, 2023

github-actions bot commented Jan 18, 2024

github-actions bot commented Jan 25, 2024

IsaakKrut commented Apr 4, 2024

tenthe commented Apr 5, 2024

pambrocio commented Apr 25, 2024

tenthe commented Apr 26, 2024

tenthe commented Apr 29, 2024

bossenti Apr 5, 2024

bossenti Apr 5, 2024

	var e = getProcessingElementParameterExtractor(processor, userConfiguration);
	var extractor = getProcessingElementParameterExtractor(processor, userConfiguration);

	// vlaidate that the processor is stopped correctly
	// validate that the processor is stopped correctly

feat(#2374): Add suggestion to test processing elements #2375

feat(#2374): Add suggestion to test processing elements #2375

Conversation

tenthe commented Dec 27, 2023

Purpose

Proposal

Remarks

github-actions bot commented Jan 18, 2024

github-actions bot commented Jan 25, 2024

IsaakKrut commented Apr 4, 2024

tenthe commented Apr 5, 2024

pambrocio commented Apr 25, 2024

tenthe commented Apr 26, 2024

tenthe commented Apr 29, 2024

bossenti Apr 5, 2024

Choose a reason for hiding this comment

bossenti Apr 5, 2024

Choose a reason for hiding this comment