Dse netty 4.1.34.final #18

sbtourist · 2019-11-21T10:52:33Z

No description provided.

… version. (netty#8289) Motivation: 6ed7c6c added a test which blindly assumed we can use a KeyManagerFactory all the time. This is only true if have OpenSSL 1.0.2 or later, which may not be the case. Modifications: Only use KeyManagerFactory in test if the OpenSSL version does support it. Result: More robust tests.

…ows an UnsupportedOperationException. (netty#8290) Motivation: 6ed7c6c added support for ExtendedOpenSslSession but we did not override getStatusResponses(). This lead to test failures on java9. Modifications: Implement ExtendedOpenSslSession.getStatusResponses() so it just returns an empty list. Result: Test pass again on Java9.

…tty#8286) Motivation: Get charset from Content-Type header even it contains multiple parameters. Modification: Extract charset value from the charset parameter if it is not last. Result: Fixes netty#8273

Motivation: Conscrypt 1.3.0 was just released and adds support for TLSv1.3 Modifications: Update to 1.3.0 Result: Use latest conscrypt during build / test.

Motivation: If you encode a SOCKS5 message like new DefaultSocks5CommandResponse(FAILURE, DOMAIN, "", 0) you correctly get a result of 05010003000000. But if the bndAddr is null, for example like new DefaultSocks5CommandResponse(FAILURE, DOMAIN) the encoded result is 0501000301000000 which means the domain name has a length of one and consists of a 0-byte. Modification: With this commit it is also correctly encoded as a string of 0 length. Result: Correctly encode empty SOCKS5 address

…enceCountedOpenSslEngine (netty#8297) Motivation: Java9 added getStatusResponses() to ExtendedSSLSession which we should correctly support when possible. Modifications: Implement the method correctly. Result: More complete and correct implementation.

Motivation: I noticed that we had some errors showing up in a test (which did not fail it tho) because we tried to full-fill the promise multiples times. Modifications: Use trySuccess(...) as we may produce multiple exceptions. Result: Less errors during test-run.

Motivation: In theory our estimation of the needed buffer could be off and so we need to ensure we grow it if there is no space left. Modifications: Ensure we grow the buffer if there is no space left in there but we still have data to deflate. Result: Correctly deflate data in all cases.

Motivation: We should use final Java11 release during builds. Modifications: Update to final Java11 release Result: Use latest release.

Motivation: We need to release the ReferenceCountedSslContext to eliminate resource leaks. Reported in https://garage.netty.io/teamcity/viewLog.html?buildId=33353&buildTypeId=netty_build_oraclejdk8&tab=buildLog#_focus=157264. Modifications: Call release on the SslContext instances. Result: No more leaks in tests.

Motivation: First EA releases of Java12 are out we should be able to compile with these and run tests. Modifications: Add maven profile for java12. Result: Be able to use Java12

netty#8318) (netty#8319) Motivation Applications should not depend on internal packages with Java 9 and later. This cause a warning now, but will break in future versions of Java. Modification This change adds methods to UnixResolverDnsServerAddressStreamProvider (following after netty#6844) that parse /etc/resolv.conf for domain and search entries. Then DnsNameResolver does not need to rely on sun.net.dns.ResolverConfiguration to do this. Result Fixes netty#8318. Furthermore, at least in my testing with Java 11, this also makes multiple search entries work properly (previously I was only getting the first entry).

…in buffer. (netty#8325) Motivation: We need to ensure the Cumulator always releases the input buffer if it can not take over the ownership of it as otherwise it may leak. Modifications: - Correctly ensure the buffer is always released. - Add unit tests. Result: Ensure buffer is always released.

Motivation: We should add some command to be able to run all tests with leak detection enabled. This will then be used on the CI during PR builds. Modifications: Add new docker-compose config to run with leak-detection enabled. Result: Easy way to enable leak detection while running tests via docker.

Motivation: 4d14586 did fix some leaks in SniClientTest but missed the ones in SniClientJava8TestUtil. Modifications: Correctly release SslContext. Result: No more leaks in SNI tests.

Motivation: The first EA builds for Java12 are released so we should allow to run with these in our docker-compose setup. Modifications: Add docker-compose configs for Java12. Result: Be able to run easily with Java12 as well.

…etty#8314) * Add cache for CNAME mappings resolved during lookup of DNS entries. Motivation: If the CNAMEd hostname is backed by load balancing component, typically the final A or AAAA DNS records have small TTL. However, the CNAME record itself is setup with longer TTL. For example: * x.netty.io could be CNAMEd to y.netty.io with TTL of 5 min * A / AAAA records for y.netty.io has a TTL of 0.5 min In current Netty implementation, original hostname is saved in resolved cached with the TTL of final A / AAAA records. When that cache entry expires, Netty recursive resolver sends at least two queries — 1st one to be resolved as CNAME record and the 2nd one to resolve the hostname in CNAME record. If CNAME record was cached, only the 2nd query would be needed most of the time. 1st query would be needed less frequently. Modifications: Add a new CnameCache that will be used to cache CNAMEs and so may reduce queries. Result: Less queries needed when CNAME is used.

…netty#8316) * Use AuthoritativeDnsServerCache for creating the new redirect stream. Motivation: At the moment if a user wants to provide custom sorting of the nameservers used for redirects it needs to be implemented in two places. This is more complicated as it needs to be. Modifications: - Just delegate to the AuthoritativeDnsServerCache always as we fill it before we call newRedirectDnsServerStream anyway. Result: Easier way for the user to implement custom sorting.

…d with the OpenSSL provider. (netty#8307) Motivation: When a X509TrustManager is used while configure the SslContext the JDK automatically does some extra checks during validation of provided certs by the remote peer. We should do the same when our native implementation is used. Modification: - Automatically wrap a X509TrustManager and so do the same validations as the JDK does. - Add unit tests. Result: More consistent behaviour. Fixes netty#6664.

…t. (netty#8333) Motivation: a208f6d added a testcase which uses hostname validation which may not be supported by OpenSSL depending on the version that is used. We should check first before we try to use it. Modifications: Add assumeTrue(...) check to ensure hostname validation is supported before trying to run the test. Result: No more test-failures on OpenSSL versions < 1.0.2.

…ng server-side and support more methods of ExtendedSSLSession. (netty#8283) Motivation: Before when on server-side we just called the X509KeyManager methods when handshake() was called the first time which is not quite correct as we may not have received the full SSL hello / handshake and so could not extra for example the SNI hostname that was requested. OpenSSL exposes the SSL_CTX_set_cert_cb function which allows to set a callback which is executed at the correct moment, so we should use it. This also allows us to support more methods of ExtendedSSLSession easily. Modifications: - Make use of new methods exposed by netty-tcnative since netty/netty-tcnative#388 to ensure we select the key material at the correct time. - Implement more methods of ExtendedOpenSslSession - Add unit tests to ensure we are able to retrieve various things on server-side in the X509KeyManager and so verify it is called at the correct time. - Simplify code by using new netty-tcnative methods. Result: More correct implementation for server-side usage and more complete implemented of ExtendedSSLSession.

Motivation The EpollChannelConfig (same for KQueues) and its subclasses repeatetly declare their own channel field which leads to a 3x repetition for each config instance. Given the fields are protected or package-private it's exposing the code code to "field hiding" bugs. Modifications Use the the existing protected channel field from the DefaultChannelConfig class and simply cast it when needed. Result Fixes netty#8331

…ailure (netty#8332) Motivation: When writing an HTTP/2 HEADERS with END_STREAM=1, the application expects the stream to be closed afterward. However, the write can fail locally due to HPACK encoder and similar. When that happens we need to make sure to issue a RST_STREAM otherwise the stream can be closed locally but orphaned remotely. The RST_STREAM is typically handled by Http2ConnectionHandler.onStreamError, which will only send a RST_STREAM if that stream still exists locally. There are two possible flows for trailers, one handled immediately and one going through the flow controller. Previously they behaved differently, with the immedate code calling the error handler after closing the stream. The immediate code also used a listener for calling closeStreamLocal while the flow controlled code did so immediately after the write. The two code paths also differed in their VoidChannelPromise handling, but both were broken. The immediate code path called unvoid() only if END_STREAM=1, however it could always potentially add a listener via notifyLifecycleManagerOnError(). And the flow controlled code path unvoided incorrectly, changing the promise completion behavior. It also passed the wrong promise to closeStreamLocal() in FlowControlledBase. Modifications: Move closeStreamLocal handling after calls to onError. This is the primary change. Now call closeStreamLocal immediately instead of when the future completes. This is the more likely correct behavior as it matches that of DATA frames. Fix all the VoidChannelPromise handling. Result: Http2ConnectionHandler.onStreamError sees the same state as the remote and issues a RST_STREAM, properly cleaning up the stream.

Motivation: Add an option (through a SelectStrategy return code) to have the Netty event loop thread to do busy-wait on the epoll. The reason for this change is to avoid the context switch cost that comes when the event loop thread is blocked on the epoll_wait() call. On average, the context switch has a penalty of ~13usec. This benefits both: The latency when reading from a socket Scheduling tasks to be executed on the event loop thread. The tradeoff, when enabling this feature, is that the event loop thread will be using 100% cpu, even when inactive. Modification: Added SelectStrategy option to return BUSY_WAIT Epoll loop will do a epoll_wait() with no timeout Use pause instruction to hint to processor that we're in a busy loop Result: When enabled, minimizes impact of context switch in the critical path

) Motivation: No need in varargs, the method always accepts array. Modification: ... replaced with []

Motivation: Unless the 'io.netty.noKeySetOptimization' system property is set, registering a SelectableChannel instance to a NioEventLoop results in a ClassCastException: io.netty.channel.nio.SelectedSelectionKeySetSelector cannot be cast to java.nio.channels.spi.AbstractSelector Modifications: Instead of 'selector', pass 'unwrappedSelector' to SelectableChannel. Result: It is possible to register a SelectableChannel instance without setting the 'io.netty.noKeySetOptimization' system property.

…ty#8347) Motivation: Avoid creating any StringBuilder instance if ByteBufInputStream::readLine isn't used Modifications: The StringBuilder instance is lazy allocated on demand and are added new test case branches to address the increased complexity of ByteBufInputStream::readLine Result: Reduced GC activity if ByteBufInputStream::readLine isn't used

Motivation: Seems like IntegerHolder counterHashCode field is the very old legacy field that is no longer used. Should be marked as deprecated and removed in the future versions. Modification: IntegerHolder class, InternalThreadLocalMap.counterHashCode() and InternalThreadLocalMap.setCounterHashCode(IntegerHolder counterHashCode) are now deprecated.

…etty#8900) Motivation: We should run a CI job using J9 to ensure netty also works when using different JVMs. Modifications: - Adjust PooledByteBufAllocatorTest to be able to complete faster when using a JVM which takes longer when joining Threads (this seems to be the case with J9). - Skip UDT tests on J9 as UDT is not supported there. Result: Be able to run CI against J9.

Motivation: To ensure Netty works on different JVMs we should also run tests on the CI with these. Modifications: Add docker-compose config to run build with OpenJ9 JVM Result: Ensure Netty works with different JVMs

…engine is destroyed (netty#8905) Motivation: We must only remove ReferenceCountedOpenSslEngine from OpenSslEngineMap when engine is destroyed as the verifier / certificate callback may be called multiple times when the remote peer did initiate a renegotiation. If we fail to do so we will cause an NPE like this: ``` 13:16:36.750 [testsuite-oio-worker-5-18] DEBUG i.n.h.s.ReferenceCountedOpenSslServerContext - Failed to set the server-side key material java.lang.NullPointerException: null at io.netty.handler.ssl.OpenSslKeyMaterialManager.setKeyMaterialServerSide(OpenSslKeyMaterialManager.java:69) at io.netty.handler.ssl.ReferenceCountedOpenSslServerContext$OpenSslServerCertificateCallback.handle(ReferenceCountedOpenSslServerContext.java:212) at io.netty.internal.tcnative.SSL.readFromSSL(Native Method) at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.readPlaintextData(ReferenceCountedOpenSslEngine.java:575) at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1124) at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1236) at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1279) at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:217) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1330) at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1237) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.oio.AbstractOioByteChannel.doRead(AbstractOioByteChannel.java:170) at io.netty.channel.oio.AbstractOioChannel$1.run(AbstractOioChannel.java:40) at io.netty.channel.ThreadPerChannelEventLoop.run(ThreadPerChannelEventLoop.java:69) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:834) ``` While the exception is kind of harmless (as we will reject the renegotiation at the end anyway) it produces some noise in the logs. Modifications: Don't remove engine from map after handshake is complete but wait for it to be removed until the engine is destroyed. Result: No more NPE and less noise in the logs.

…netty#8904) Motivation: Netty is very widely used which can lead to a lot of pain when we break API / ABI. We should make use japicmp-maven-plugin during the build to verify we do not introduce breakage by mistake. Modifications: - Add japicmp-maven-plugin to the build process - Fix a method signature change in HttpProxyHandler that was flagged as a possible problem. Result: Ensure no API/ABI breakage accour between releases.

…y name. (netty#8913) Motivation: 2bb9f64 introduced a change which made it possible to use different shaded versions of netty-tcnative on the classpath. This only partly worked as we did not correctly handled the case when os / arch is part of the library name (which is the case when netty-tcnative-boringssl-static is used with the uber jar). Modifications: - If patching the ID failed we retry again with the os / arch stripped - Add unit tests to verify that patching ID now works with and without os / arch as suffix. Result: Using multiple shaded version of netty-tcnative-boringssl-static on MacOS works.

…tty#8859) Motivation: SSLEngine API has a notion of tasks that may be expensive and offload these to another thread. We did not support this when using our native implementation but can now for various operations during the handshake. Modifications: - Support offloading tasks during the handshake when using our native SSLEngine implementation - Correctly handle the case when NEED_TASK is returned and nothing was consumed / produced yet Result: Be able to offload long running tasks from the EventLoop when using SslHandler with our native SSLEngine.

…y#8914) Motivation: Simple rules: * close the connection when sending any error * specify "Connection: close" header when closing the connection * successful responses should keep the connection intact when otherwise is not requested by the client Modifications: * "send response and cleanup the connection" logic moved to a helper * for all successful responses set "Content-Lenght" header * do not specify "Connection: Keep-Alive" header as far it's a default for HTTP/1.1 * set "Connection: close" header when necessary Result: Keep-Alive connections management is inlined with RFCs.

…tions. (netty#8919) Motivation: In the past we found a lot of SSL related bugs because of the interopt tests we have in place between different SSLEngine implementations. We should have as many of these interopt tests as possible for this reason. Modifications: - Add interopt tests between Conscrypt and OpenSSL SSLEngine implementations Result: More tests for SSL.

@jayv

netty#8921) * Do not use GetPrimitiveArrayCritical(...) due multiple not-fixed bugs related to GCLocker Motivation: GetPrimitiveArrayCritical(...) may cause multiple not-fixed bugs related to the GCLocker while there is little gain for our use-case. We should just use GetByteArrayRegion(...) and copy into a small on-stack buffer. See also: - https://shipilev.net/jvm/anatomy-quarks/9-jni-critical-gclocker/#_g1 - https://bugs.openjdk.java.net/browse/JDK-8048556 - https://bugs.openjdk.java.net/browse/JDK-8057573 - https://bugs.openjdk.java.net/browse/JDK-8057586 Special thanks to @jayv @shipilev @apangin for the pointers. Modifications: Replace GetPrimitiveArrayCritical(...) with GetByteArrayRegion(...) Result: Less risks hitting GCLocker related bugs.

netty#8922) Motivation: When HttpContentDecoder (and so HttpContentDecompressor) does not produce any message we need to make sure it calls ctx.read() if auto read is false to not stale. Modifications: - Keep track if we need to call ctx.read() or not - Add unit test Result: Fixes netty#8915.

netty#8918) Motivation: The SSLSession that is returned by SSLEngine.getHandshakeSession() must be able to provide the local certificates when the TrustManager is invoked on the server-side. Modifications: - Correctly return the local certificates - Add unit test Result: Be able to obtain local certificates from handshake SSLSession during verification on the server side.

Motivation: While looking at hpack header-processing hotspots I noticed some low level too-big-to-inline methods which can be shrunk. Modifications: Reduce bytecode size and/or runtime operations used for the following methods: PlatformDependent0.equals(byte[], ...) PlatformDependent0.equalsConstantTime(byte[], ...) PlatformDependent0.hashCodeAscii(byte[],int,int) PlatformDependent.hashCodeAscii(CharSequence) Result: Existing benchmarks show decent improvement Before Benchmark (size) Mode Cnt Score Error Units HpackUtilBenchmark.newEquals SMALL thrpt 5 17200229.374 ± 1701239.198 ops/s HpackUtilBenchmark.newEquals MEDIUM thrpt 5 3386061.629 ± 72264.685 ops/s HpackUtilBenchmark.newEquals LARGE thrpt 5 507579.209 ± 65883.951 ops/s After Benchmark (size) Mode Cnt Score Error Units HpackUtilBenchmark.newEquals SMALL thrpt 5 29221527.058 ± 4805825.836 ops/s HpackUtilBenchmark.newEquals MEDIUM thrpt 5 6556251.645 ± 466115.199 ops/s HpackUtilBenchmark.newEquals LARGE thrpt 5 879828.889 ± 148136.641 ops/s Before Benchmark (size) Mode Cnt Score Error Units PlatformDepBench.unsafeBytesEqual 4 avgt 10 4.263 ± 0.110 ns/op PlatformDepBench.unsafeBytesEqual 10 avgt 10 5.206 ± 0.133 ns/op PlatformDepBench.unsafeBytesEqual 50 avgt 10 8.160 ± 0.320 ns/op PlatformDepBench.unsafeBytesEqual 100 avgt 10 13.810 ± 0.751 ns/op PlatformDepBench.unsafeBytesEqual 1000 avgt 10 89.077 ± 7.275 ns/op PlatformDepBench.unsafeBytesEqual 10000 avgt 10 773.940 ± 24.579 ns/op PlatformDepBench.unsafeBytesEqual 100000 avgt 10 7546.807 ± 110.395 ns/op After Benchmark (size) Mode Cnt Score Error Units PlatformDepBench.unsafeBytesEqual 4 avgt 10 3.337 ± 0.087 ns/op PlatformDepBench.unsafeBytesEqual 10 avgt 10 4.286 ± 0.194 ns/op PlatformDepBench.unsafeBytesEqual 50 avgt 10 7.817 ± 0.123 ns/op PlatformDepBench.unsafeBytesEqual 100 avgt 10 11.260 ± 0.412 ns/op PlatformDepBench.unsafeBytesEqual 1000 avgt 10 84.255 ± 2.596 ns/op PlatformDepBench.unsafeBytesEqual 10000 avgt 10 591.892 ± 5.136 ns/op PlatformDepBench.unsafeBytesEqual 100000 avgt 10 6978.859 ± 285.043 ns/op

[maven-release-plugin] copy for tag netty-4.1.34.Final

Jenkins job that publishes netty artifacts is splitted in two subjobs. First publishes OSX artifacts, seconds publishes all other artifacts. The former depends on the latter but this dependency is not reflected in jenkins. During a first run OSX subjob fails due to unresolved dependencies (artifacts published by the second subjob). Add -U to workaround caching "unresolved dependencies".

Docker setup

Motivation: The special case fixed in netty#8497 also requires that we keep a derived slice when trimming components in place, as done by the capacity(int) and discardReadBytes() methods. Modifications: Ensure that we keep a ref to trimmed components' original retained slice in capacity(int) and discardReadBytes() methods, so that it is released properly when the they are later freed. Add unit test which fails prior to the fix. Result: Edge case leak is eliminated.

sbtourist

Excellent job, just left a few minor comments to address.

sbtourist · 2019-11-21T11:25:42Z

common/src/main/java/io/netty/util/internal/PlatformDependent.java

-        for (;;) {
-            long usedMemory = DIRECT_MEMORY_COUNTER.get();
-            long newUsedMemory = usedMemory + capacity;
+        if (DIRECT_MEMORY_COUNTER != null) {


We've made sure DIRECT_MEMORY_COUNTER is always non-null, so this check is confusing.

sbtourist · 2019-11-21T11:30:24Z

common/src/main/java/io/netty/util/internal/PlatformDependent.java

@@ -658,15 +673,13 @@ public static void freeDirectNoCleaner(ByteBuffer buffer) {
    }

    private static void incrementMemoryCounter(int capacity) {
-        for (;;) {


I'm on the fence about this change: moving from a CAS loop to the double addAndGet() might be faster, but by doing so, a single big-ish allocation which goes over the limit could cause concurrent small allocations to fail as well before the memory limit is brought back below threshold via DIRECT_MEMORY_COUNTER.addAndGet(-capacity).

Agreed. I'll revert.

sbtourist · 2019-11-22T12:32:18Z

common/src/main/java/io/netty/util/concurrent/AbstractScheduledEventExecutor.java

@@ -78,7 +78,7 @@ protected void cancelScheduledTasks() {
        }

        final ScheduledFutureTask<?>[] scheduledTasks =
-                scheduledTaskQueue.toArray(new ScheduledFutureTask<?>[scheduledTaskQueue.size()]);
+                scheduledTaskQueue.toArray(new ScheduledFutureTask<?>[0]);


While you're at this, could you make scheduledTaskQueue protected? See DB-3884.

Will do. Just out of curiosity, how will this help in DB-3884?

Thanks. No final plans yet, but we might need to peek at the scheduled queue ourselves.

Review remark fixes are pushed to a separate PR #19

- Reason for reverting incrementMemoryCounter implementation (from Sergio's comment): "..moving from a CAS loop to the double addAndGet() might be faster, but by doing so, a single big-ish allocation which goes over the limit could cause concurrent small allocations to fail as well before the memory limit is brought back below threshold via DIRECT_MEMORY_COUNTER.addAndGet(-capacity)" - scheduledTaskQueue is now protected to simplify DB-3884 "we might need to peek at the scheduled queue ourselves"

DSP-19407 after upgrade review remarks

…netty#9492) Motivation: `HttpObjectDecoder` pre-checks that it doesn't request characters outside of the `AppendableCharSequence`'s length. `0` is always allowed because the minimal length of `AppendableCharSequence` is `1`. We can legally skip index check by using `AppendableCharSequence.charAtUnsafe(int)` in all existing cases in `HttpObjectDecoder`. Modifications: - Use `AppendableCharSequence.charAtUnsafe(int)` instead of `AppendableCharSequence.charAt(int)` in `HttpObjectDecoder`. Result: No unnecessary index checks in `HttpObjectDecoder`. (cherry picked from commit 85fcf4e)

…30#section-3.2.4 (netty#9585) Motivation: When parsing HTTP headers special care needs to be taken when a whitespace is detected in the header name. Modifications: - Ignore whitespace when decoding response (just like before) - Throw exception when whitespace is detected during parsing - Add unit tests Result: Fixes netty#9571 (cherry picked from commit 39cafcb)

) Motivation: Technical speaking its valid to have http headers with no values so we should support it. That said we need to detect if these are "generated" because of an "invalid" fold. Modifications: - Detect if a colon is missing when parsing headers. - Add unit test Result: Fixes netty#9866 (cherry picked from commit a7c18d4)

…-length and transfer-encoding: chunked header when using HTTP/1.1 (netty#9865) Motivation: RFC7230 states that we should not accept multiple content-length headers and also should not accept a content-length header in combination with transfer-encoding: chunked Modifications: - Check for multiple content-length headers and if found mark message as invalid - Check if we found a content-length header and also a transfer-encoding: chunked and if so mark the message as invalid - Add unit test Result: Fixes netty#9861 (cherry picked from commit 8494b04)

Compared against 4.1.34.2.dse, this tag cherry-picks upstream commits that fixed bugs in HttpObjectDecoder/HttpRequestDecoder, plus two intermediate refactoring commits that indirectly affect those bugfix commits. What follows is a list of PR links, issue links, CVE links, and hashes associated with the cherry-picked commits. Verify we do not receive multiple content-length headers or a content-length and transfer-encoding: chunked header when using HTTP/1.1 (netty#9865) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-7238 netty#9861 netty#9865 8494b04 Detect missing colon when parsing http headers with no value (netty#9871) https://nvd.nist.gov/vuln/detail/CVE-2019-20444 netty#9866 netty#9871 a7c18d4 Fix typos in javadocs (netty#9527) skipped Correctly handle whitespaces in HTTP header names as defined by RFC7230#section-3.2.4 (netty#9585) https://nvd.nist.gov/vuln/detail/CVE-2019-16869 netty#9571 netty#9585 39cafcb Use `AppendableCharSequence.charAtUnsafe(int)` in `HttpObjectDecoder` (netty#9492) netty#9492 85fcf4e

DB-4068 cherry-pick upstream HttpRequest/ObjectDecoder fixes (4.1.34/bdp master)

normanmaurer and others added 30 commits September 14, 2018 19:01

Fix get charset from content-type header with multiple parameters (ne…

2ab3e13

…tty#8286) Motivation: Get charset from Content-Type header even it contains multiple parameters. Modification: Extract charset value from the charset parameter if it is not last. Result: Fixes netty#8273

Update to Conscrypt 1.3.0 (netty#8296)

6872753

Motivation: Conscrypt 1.3.0 was just released and adds support for TLSv1.3 Modifications: Update to 1.3.0 Result: Use latest conscrypt during build / test.

Update to final Java11 release (netty#8320)

60a7ece

Motivation: We should use final Java11 release during builds. Modifications: Update to final Java11 release Result: Use latest release.

Add profile to be able to compile on java12 (netty#8321)

618a98f

Motivation: First EA releases of Java12 are out we should be able to compile with these and run tests. Modifications: Add maven profile for java12. Result: Be able to use Java12

Fix leak in SniClientJava8TestUtil (netty#8326)

652650e

Motivation: 4d14586 did fix some leaks in SniClientTest but missed the ones in SniClientJava8TestUtil. Modifications: Correctly release SslContext. Result: No more leaks in SNI tests.

Add docker-compose config to run with Java12 (netty#8327)

70efd25

Motivation: The first EA builds for Java12 are released so we should allow to run with these in our docker-compose setup. Modifications: Add docker-compose configs for Java12. Result: Be able to run easily with Java12 as well.

[maven-release-plugin] prepare release netty-4.1.30.Final

3a9ac82

[maven-release-plugin] prepare for next development iteration

2d7cb47

remove unnecessary vararg argument in PooledByteBufAllocator (netty#8338

6cebb60

) Motivation: No need in varargs, the method always accepts array. Modification: ... replaced with []

normanmaurer and others added 21 commits March 1, 2019 06:47

Add docker-compose config to run build with OpenJ9 JVM (netty#8903)

ef3e98d

Motivation: To ensure Netty works on different JVMs we should also run tests on the CI with these. Modifications: Add docker-compose config to run build with OpenJ9 JVM Result: Ensure Netty works with different JVMs

[maven-release-plugin] prepare release netty-4.1.34.Final

ff7a9fa

Merge tag 'netty-4.1.34.Final' into dse-netty-4.1.34.Final-temp

1b8965f

[maven-release-plugin] copy for tag netty-4.1.34.Final

fix errors after netty 4.1.34 merge

dec3128

Version bump to 4.1.34.1.dse

09e434c

publishing fixes (repo has changed)

44a09b5

Build from centos 6 to support old glibc DSP-18603

11fbd25

Docker setup

DSP-19407 bump netty version to temp 4.1.34.2

136266b

sbtourist commented Nov 22, 2019

View reviewed changes

jtgrabowski and others added 8 commits December 10, 2019 15:38

Merge pull request #19 from riptano/DSP-19407-upgrade-review-remarks

1c6673a

DSP-19407 after upgrade review remarks

Merge pull request #21 from dalaro/dse-netty-4.1.34.Final-with-DB-4068

e50eb60

DB-4068 cherry-pick upstream HttpRequest/ObjectDecoder fixes (4.1.34/bdp master)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dse netty 4.1.34.final #18

Dse netty 4.1.34.final #18

sbtourist commented Nov 21, 2019

sbtourist left a comment

sbtourist Nov 21, 2019

jtgrabowski Dec 6, 2019

sbtourist Nov 21, 2019

jtgrabowski Dec 6, 2019

sbtourist Nov 22, 2019

jtgrabowski Dec 6, 2019

sbtourist Dec 9, 2019

jtgrabowski Dec 10, 2019

Dse netty 4.1.34.final #18

Are you sure you want to change the base?

Dse netty 4.1.34.final #18

Conversation

sbtourist commented Nov 21, 2019

sbtourist left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment