0.93.0

[0.93.0]

What's Changed

Enhancement 🌟

• [#2092]: Remove magic HTTP numbers in StreamPipes
• [#2056]: Make email templates configurable
• [#2032]: Provide endpoint to get measurement counts from core
• [#2031]: style: Add last message info to adapter overview
• [#1992]: Migration of Kafka source configuration.
• [#1980]: Arrays are not supported for S7 PLCs
• [#1906]: Revive streampipes-maven-plugin to auto-generate
  pipeline element documentation
• [#1875]: Connect: Order measurement units by name
• [#1814]: Integrate extensions service discovery & configuration
  management into core
• [#1716]: Enable creating CouchDB attachments for images
• [#1688]: New Processor: Round Numeric Values
• [#1662]: Support asynchronous browsing in OPC-UA adapter
• [#1592]: Connect IO-Link Sensor Data into StreamPipes
• [#1374]: Convenient columns query parameter for data lake
  measure
• [#1103]: Support Python 3.11 in python client

Bug fixes 🧰

• [#2191]: fix: tooltip in asset overview
• [#2146]: fix(#2002) Retry service registration in case services are
  removed be…
• [#2166]: Protected names are not sanitized correctly in Data Lake
  Sink / Influx sink
• [#2165]: Update 0.92.0 -> 0.93.0 of Machine Data Simulator
  not working
• [#2112]: Changes on messaging layer configuration on UI not
  persisted.
• [#2044]: Docker compose build error
• [#2024]: fix: make data retrieval of IOLink sensor more robust
• [#1992]: Migration of Kafka source configuration.
• [#1983]: Logo image broken in Footer
• [#1956]: NPE in ConsulConfigMigration
• [#1938]: Datetime selector in Data Explorer has issues with 12 am
• [#1934]: Improve adapter started dialog in StreamPipes connect
• [#1876]: Connect: Form validation in schema editor
• [#1834]: [hotfix] Fix MDC layout issue in permission dialog
• [#1829]: [hotfix] Fix layout issues and validation in data explorer
• [#1794]: Aggregation field in data explorer widget is broken
• [#1770]: Wrong base image in Maven archetypes
• [#1769]: The dashboard fails to load the element whose name
  contiains '/'
• [#1741]: The status light widget in the live dashboard is broken
• [#1713]: OPC UA NullPointer Exception when Node Description is
  Missing
• [#1642]: Data Lake default export period does not work
• [#1637]: Schema guessing from file is currently not implemented
  in HTTP Server source
• [#1629]: fix: kafka consumer data loss promble
• [#1597]: apachestreampipes/sources-vehicle-simulator:
  0.92.0-SNAPSHOT not found
• [#1546]: Failed to upgrade the helm chart
• [#1533]: Notification counter is not reset
• [#1481]: URL Dereferencing Processor NotSerializableException

Breaking Change 💣

• [#2143]: refactor(#2128): deprecate legacy adapters
• [#2088]: Remove module streampipes-logging
• [#2066]: refactor: remove legacy demo resources
• [#1912]: Remove python wrapper
• [#1583]: Remove CumSum Pipeline Element
• [#1289]: Harmonize data set and data stream API

Deprecation ⚠️

• [#2143]: refactor(#2128): deprecate legacy adapters
• [#1640]: feature: retrieve credentials from SP environment
  variables

Documentation & Website 📚

• [#2143]: refactor(#2128): deprecate legacy adapters
• [#2138]: Use os.environ dictionary to set environment variables.
• [#2069]: refactor: Add individual connector modules for adapters
  and sinks
• [#2067]: Check references for watertank simulator and vehicle
  simulator on the website
• [#2066]: refactor: remove legacy demo resources
• [#1983]: Logo image broken in Footer
• [#1978]: Add deployment of Prometheus and Grafana to K8s
• [#1955]: Introduce Quickstart deployment mode
• [#1945]: Add configuration hint for Kafka users.
• [#1912]: Remove python wrapper
• [#1906]: Revive streampipes-maven-plugin to auto-generate
  pipeline element documentation
• [#1844]: chore: add Poetry badge to our README
• [#1820]: Add Pulsar's Messaging Layer to a Helm Deployment
• [#1817]: chore: introduce poetry as dependency management tool
• [#1733]: feature: introduce admonitions to warn about dependency
  issue in docs
• [#1694]: chore: improve metadata for repository
• [#1640]: feature: retrieve credentials from SP environment
  variables

Dependency Updates 📦

• [#2177]: deps: update Active MQ due to CVE
• [#2140]: chore(deps-dev): bump eslint from 8.37.0 to 8.53.0 in /ui
• [#2127]: chore(deps-dev): bump
  @angular-eslint/eslint-plugin-template from 15.2.1 to 16.2.0 in /ui
• [#2126]: chore(deps): bump org.mockito:mockito-core from 5.6.0 to
  5.7.0
• [#2125]: chore(deps): remove org.wildfly.common:wildfly-common
• [#2119]: chore(deps): bump com.google.protobuf:protobuf-java from
  3.24.0 to 3.25.0
• [#2113]: chore(deps-dev): bump webpack from 5.88.2 to 5.89.0 in /ui
• [#2091]: chore(deps-dev): bump browserify-sign from 4.2.1 to 4.2.2
  in /ui
• [#2087]: chore(deps): bump com.rabbitmq:amqp-client from 5.19.0 to
  5.20.0
• [#2085]: chore(deps-dev): bump jasmine-core from 4.6.0 to 5.1.1 in
  /ui
• [#2069]: refactor: Add individual connector modules for adapters
  and sinks
• [#2066]: refactor: remove legacy demo resources
• [#2048]: chore(deps-dev): bump assert from 2.0.0 to 2.1.0 in /ui
• [#2038]: chore(deps-dev): bump @babel/traverse from 7.22.5 to
  7.23.2 in /ui
• [#2036]: chore(deps): bump plotly.js from 2.22.0 to 2.26.2 in /ui
• [#2035]: chore(deps): bump org.apache.inlong:tubemq-client from
  1.7.0 to 1.9.0
• [#2027]: chore(deps-dev): bump @types/jasmine from 4.3.1 to 5.1.0
  in /ui
• [[#2019](https://github.com/apache/streampipes/pu...

0.92.0

[0.92.0]

What's Changed

Enhancement 🌟

• [#1564]: Add feature to assign labels to StreamPipes resources
• [#1522]: Return 404 if requested pipeline does not exist
• [#1520]: Map http 404 to Java Optional.empty in Java Client
• [#1518]: Compiling maven submodule with checkstyle check
• [#1467]: Implement create method in Java client PipelineApi
• [#1405]: Enable search for Python docs
• [#1339]: Support backend-only development mode (#1327)
• [#1319]: chore(ui): provide some more references for support in the 'about' view
• [#1307]: Geometry Validation processor
• [#1296]: Swinging Door Trending (SDT) Filter Processor
• [#1272]: Buffer Processor
• [#1261]: Add Python changes to release notes
• [#1259]: Verify authentication on startup of Python client
• [#1113]: Add buttons to start and stop all adapters
• [#1107]: Build Github worfklow with OSV scanner
• [#1101]: Extend StreamPipes API by a health-check endpoint

Bug fixes 🧰

• [#1547]: standalone dashborad empty
• [#1527]: Fix logo url of email templates
• [#1501]: [hotfix] Fix handling of count queries in data explorer
• [#1479]: Truncate does not work in data lake configuration
• [#1439]: Past data exists twice in raw widget
• [#1391]: Data Explorer filter doesn't work for boolean and false value
• [#1333]: Endpoint to receive pipelines is broken
• [#1329]: Data Lake measurements endpoint returns no data series when no data between startDate and endDate
• [#1327]: Failed to start local backend enviroment
• [#1308]: [CI] label-pr workflow fails on fork-based PRs
• [#1291]: Can't aggregate calculation results of the Math processing element
• [#1199]: UI Semantik Type field is too small to see full term
• [#793]: Empty string as API token name

Breaking Change 💣

• [#1040]: Change Event Runtime Name of geometry field

Deprecation ⚠️

• [#1115] IMPORTANT In the future we will remove support for data set adapters

Documentation & Website 📚

• [#1623]: chore: mark streampipes-wrapper-python as deprecated
• [#1535]: Add vulnerability report
• [#1442]: Extend Python docs with dark mode
• [#1415]: docEnhancement in some Hyperlinks.
• [#1411]: Create custom error page for Python docs
• [#1405]: Enable search for Python docs
• [#1327]: Failed to start local backend enviroment
• [#1326]: chore(docs): add information on how to develop the UI locally
• [#1051]: Add further badges to README of Python client

Dependency Updates 📦

• [#1623]: chore: mark streampipes-wrapper-python as deprecated
• [#1577]: Bump tubemq-client from 1.5.0 to 1.6.0
• [#1574]: Bump amqp-client from 5.16.0 to 5.17.0
• [#1571]: Remove guice from dependencies
• [#1570]: Bump annotations from 16.0.2 to 24.0.1
• [#1566]: Bump mypy from 1.2.0 to 1.3.0 in /streampipes-client-python
• [#1562]: Bump jackson-databind from 2.14.1 to 2.15.0
• [#1560]: Bump google-maps-services from 2.1.2 to 2.2.0
• [#1559]: Bump pyupgrade from 3.3.1 to 3.4.0 in /streampipes-client-python
• [#1558]: Bump types-requests from 2.29.0.0 to 2.30.0.0 in /streampipes-client-python
• [#1552]: Bump opencsv from 5.5.2 to 5.7.1
• [#1550]: Bump commons-compress from 1.22 to 1.23.0
• [#1549]: Bump engine.io and socket.io in /ui
• [#1545]: Bump consul from 1.17.6 to 1.18.0
• [#1542]: Bump jackson.version from 2.14.1 to 2.15.0
• [#1541]: Bump pre-commit from 3.2.0 to 3.3.0 in /streampipes-client-python
• [#1540]: Bump flask from 1.1.2 to 2.3.2 in /streampipes-wrapper-python
• [#1539]: chore(deps): bump several dependencies to resolve vulnerability issues
• [#1536]: Bump types-requests from 2.28.11.7 to 2.29.0.0 in /streampipes-client-python
• [#1525]: Bump mkdocs-gen-files from 0.4.0 to 0.5.0 in /streampipes-client-python
• [#1516]: Bump karma-chrome-launcher from 3.1.1 to 3.2.0 in /ui/projects/streampipes/platform-services
• [#1512]: Bump jetty-server from 10.0.10 to 10.0.14
• [#1511]: Bump spring-core from 6.0.7 to 6.0.8 & spring-bot from 3.0.5 to 3.0.6 & spring-security from 6.0.2 to 6.0.3
• [#1510]: Bump type-parser from 0.7.0 to 0.8.1
• [#1507]: Bump maven-plugin-plugin from 3.7.0 to 3.8.1
• [#1506]: Bump pandas-stubs from 1.5.2.230105 to 2.0.0.230412 in /streampipes-client-python
• [#1503]: Bump peter-evans/create-pull-request from 4 to 5
• [#1502]: Bump mkdocs-awesome-pages-plugin from 2.8.0 to 2.9.0 in /streampipes-client-python
• [#1499]: Bump log4j.version from 2.19.0 to 2.20.0
• [#1498]: Bump pytest from 7.2.1 to 7.3.0 in /streampipes-client-python
• [#1472]: Bump rdf4j.version from 3.5.0 to 3.7.7

Uncategorized ❓

• [#1532]: [hotfix] exclude versions file from pydocs update
• [#1524]: feature: add workflow that stales PRs
• [#1517]: [FEATURE] add division as operation for data harmonization rules
• [#1504]: Data streams are removed from pipelines when exported
• [#1497]: [hotfix] Move sp-split-section to shared-ui module
• [#1484]: [hotfix] Improve handling of count queries, improve table widget
• [#1478]: Distinguish between Consumer and Publisher for the broker
• [#1475]: Fulfill PEP 561 compatibility
• [#1465]: [hotfix] Fix configuration for pipeline element development
• [#1445]: Improve Log info in Processor Test
• [#1433]: update archetypes template
• [#1428]: Maintain the same directory structure for source and test
• [#1406]: Cleanup data explorer query management
• [#1385]: Remove unused features from data explorer module
• [#1383]: Introduce environment variable t...

0.91.0

Appreciation

We would like to express our sincere thanks to Котко Владислав for helping us fix a potential security vulnerability.
In February 2023, he pointed out to us that @streampipes could be a potential target for dependency confusion in NPM.
As a result, we have taken all necessary countermeasures so that there is no longer a threat.
We are very grateful for the support and professional handling.

What's Changed

Enhancement 🌟

• [#1375]: Versioning for Python docs
• [#1363]: Add support for the Kafka broker in Python
• [#1254]: Adapt data lake measure endpoint's get() method to process query parameter in Python
• [#1191]: [#1190] Apache TubeMQ (InLong) Adapter & Sink
• [#1182]: Output data streams for python functions
• [#1149]: Extend REST API by endpoint to get a specific function definition
• [#1133]: Support output streams in functions
• [#1121]: Add option for single file replay in FileStreamAdapter
• [#1099]: [hotfix] Fix failing e2e tests
• [#1096]: Improve structure of pipeline execution
• [#1091]: Reduce warnings on service startup
• [#1085]: Refactor FileStreamProtocol
• [#1081]: restrict appearance of dependabot PRs
• [#1077]: Remove legacy method getNElements in IProtocol
• [#1069]: Sp 1065
• [#1065]: Provide e2e tests to validate the different formats of generic adapters
• [#1050]: Extend metadata for python client publishing
• [#1031]: [hotfix] Remove unused message interface
• [#1026]: Fix JUnit tests and activate them in build pipeline
• [#881]: Enable dependabot for ui and bump all minor versions
• [#859]: Create PR validation workflow for streampipes-website
• [#854]: Implement StreamPipesFunctions for Python Client
• [#792]: Rename Python Client
• [#569]: Reorganize streampipes-container modules

Bug fixes 🧰

• [#1423]: Update Archetypes to reflect module structure
• [#1274]: Messaging protocol is not overridden when importing data
• [#1267]: Fix logging configuration (#1266)
• [#1266]: Fix logging configuration
• [#1146]: Session based implementation of IoTDB sink (fix issues in current JDBC based implementation)
• [#964]: CLI command env outputs success message in case of an error
• [#962]: Example code in docs for Python code is rendered as markdown
• [#959]: Siddhi processors could not be started in a pipeline
• [#878]: Fix README.md
• [#862]: fix speed calculation divide by 0 error
• [#858]: Remove checkboxes from issue templates

Breaking changes 💣

• [#958]: Extract REST resource classes from extensions services into own module
• [#957]: Use ExtensionsModelSubmitter over StandaloneModelSubmitter (#956)
• [#956]: Remove deprecated module streampipes-container-standalone
• [#569]: Reorganize streampipes-container modules

Deprecation ⚠️

• [#953]: Improve handling of Consul env variables
• [#883]: Add checkstyle to streampipes-wrapper-* modules (#820)

Documentation & Website 📚

• [#1496]: [hotfix] Adapt broken image links in Python docs
• [#1419]: docs: spring cleaning for Python docs
• [#1409]: docs: add matomo tracking to python docs
• [#1375]: Versioning for Python docs
• [#1364]: chore: fix example code & add disclaimer for doc versioning
• [#1253]: Update links in Python example notebooks
• [#1228]: chore(python-client): rename python package to streampipes
• [#1167]: refactor: remove outdated readme files
• [#962]: Example code in docs for Python code is rendered as markdown
• [#878]: Fix README.md
• [#859]: Create PR validation workflow for streampipes-website
• [#858]: Remove checkboxes from issue templates

Dependency Updates 📦

• [#1424]: Add slf4j-api to streampipes-service-base module
• [#1419]: docs: spring cleaning for Python docs
• [#1363]: Add support for the Kafka broker in Python
• [#1356]: chore: reify project url in package metadata
• [#1228]: chore(python-client): rename python package to streampipes
• [#1158]: Replace consul-client library with consul-api library
• [#1081]: restrict appearance of dependabot PRs
• [#1080]: chore: Bump @auth0/angular-jwt from 5.0.2 to 5.1.2 in /ui
• [#1060]: Upgrade Siddhi version to v5.1.27
• [#1016]: Bump Spring versions, migrate Spring Security classes (#1015)
• [#1015]: Bump Spring Boot to v3
• [#911]: chore: remove peer dependencies and use postinstall

Uncategorized ❓

• [#1263]: feature(gh-actions): create workflow that provides python docs as artifact weekly
• [#1258]: Simplify/restructure data stream generator
• [#1230]: Build GitHub workflow to deploy Python library to Pypi
• [#1229]: chore(gh-actions): configure custom label for GitHub actions
• [#1227]: chore(asf-yaml): add basic branch protection setup
• [#1223]: Harmonize handling of environment variables
• [#1220]: chore(gh-actions): improve pr labeling workflow
• [#1200]: Refactor InfluxDB adapter and sink
• [#1197]: Update actions versions in GitHub workflow
• [#1162]: Change GitHub notification settings in .asf.yaml
• [#1157]: Upgrade third-party services
• [#1147]: refactor: further clean up python data model & introduce function definition
• [#1140]: Redirect to previous view after login
• [#1132]: Update e2e tests to use FileStream adapter instead of FileSet
• [#1126]: refactor: introduce messaging endpoint for python client
• [#1114]: Create an adapter without starting it
• [#1104]: Add GitHub wofklow that tags PR
• [#1100]: chore: adapt setup of python dependencies
• [#1098]: [hotfix] add stream example to Mkdocs
• [#1088]: Python MkDocs use old StreamPipe...

0.90.0

The Apache StreamPipes community is pleased to announce Apache StreamPipes version 0.90.0!

This release closes 57 issues with a focus on bug fixes and improvements.
It brings several new adapters and data sinks for Nats, RocketMQ and Pulsar and includes many functional and performance improvements to the data explorer.

Release notes can be found here:
https://github.com/apache/streampipes/blob/release/0.90.0/RELEASE_NOTES.md
https://github.com/apache/streampipes/milestone/2?closed=1

0.70.0

The Apache StreamPipes (incubating) community is delighted to announce the release of Apache StreamPipes version 0.70.0!
Key highlights of this release include many enhancements to the data explorer for fast exploration of IoT data and a brand new asset management that allows importing and exporting of StreamPipes resources.
In addition, our new version includes more than 50 improvements and bug fixes.

Release notes can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351345&styleName=&projectId=12323520
https://github.com/apache/incubator-streampipes/blob/release/0.70.0/RELEASE_NOTES.md

0.69.0

The Apache StreamPipes (incubating) community is pleased to announce Apache StreamPipes version 0.69.0!

The most notable highlights of this release include a completely reworked data explorer for quick exploration of IoT data, improved pipeline modeling and advanced user and access rights management.
In addition, our new release includes more than 80 improvements and bug fixes.

Release notes can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12350230&styleName=Text&projectId=12323520
https://github.com/apache/incubator-streampipes/blob/release/0.69.0/RELEASE_NOTES.md

0.68.0

The Apache StreamPipes (incubating) community is pleased to announce the availability of Apache StreamPipes (incubating) 0.68.0!

The latest release brings many improvements related to health management of pipelines and pipeline monitoring, improved user guidance such as a live preview in the pipeline editor and better handling of pipeline updates as well as several new data processors and sinks such as support for NATS. In addition, the release contains several bug fixes related to K8s deployment, PLC adapters and pipeline management.

Release notes can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12323520&version=12348698
https://github.com/apache/incubator-streampipes-extensions/blob/release/0.68.0/RELEASE_NOTES.md

0.67.0

The Apache StreamPipes (incubating) community is pleased to announce the availability of Apache StreamPipes (incubating) 0.67.0!

The latest release brings major improvements in the UI. All AngularJS components are refactored to Angular. Additionally new data processors, sinks, and adapters are available and a variety of bug fixes and minor improvements.

Release notes can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12323520&version=12347026

0.66.0

The Apache StreamPipes (incubating) community is pleased to annouce the availability of Apache StreamPipes (incubating) 0.66.0!

The latest release brings major improvements to the live dashboard, the notification system, multiple new pipeline elements and StreamPipes Connect adapters, a simplified installation, more deployment options and a variety of bug fixes and minor improvements.

Release notes can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12323520&version=12347025

0.65.0-pre-asf

New features

• Added a welcome screen to the UI during startup
• Added an umbrella pipeline element to reduce memory requirements of lite version
• Bumped Flink to latest version 1.9.1
• Added CSV enrichment processor
• Added event counter processor
• Support FileStaticProperty in Pipeline Editor

Improvements

• Do not trigger cache refresh of pipeline elements during initial installation
• Websocket URL in live dashboard does not depend anymore on hostname
• Optimize Dockerfiles
• Installer now works without providing a hostname
• Optimize caching of pipeline element descriptions

Bug fixes

• Fixed a bug in the OPC-UA Adapter
• Fixed a bug that prevented error messages to be shown in the pipeline view
• Fixed a bug that cause the pipeline modification to fail