Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NULL message headers in CentOS7 on PHP5.6 #386

Open
douyux opened this issue Sep 9, 2020 · 9 comments
Open

NULL message headers in CentOS7 on PHP5.6 #386

douyux opened this issue Sep 9, 2020 · 9 comments

Comments

@douyux
Copy link

douyux commented Sep 9, 2020

when test message_headers.php in centos i got following output:

> php message_headers.php
6 messages delivered  
Got message: message 0 | Headers: none  
PHP Warning:  array_walk() expects parameter 1 to be array, unknown given in /root/test/message_headers.php on line 80  
Got message: message 1 | Headers: none  
PHP Warning:  array_walk() expects parameter 1 to be array, unknown given in /root/test/message_headers.php on line 80  
Got message: message 2 | Headers: none  
Got message: message 3 | Headers: none  
Got message: message 4 | Headers: none  
Got message: message 5 | Headers: none 

and var_dump($msg->headers) just print UNKNOWN:0

  • os: CentOS Linux release 7.6.1810 (Core)
  • kernel: 3.10.0-957.1.3.el7.x86_64
  • PHP version: 5.6.36
  • librdkafka version: v1.5.0
  • php-rdkafka version: 4.0.3
  • kafka version: 2.12-2.3.1

It's confused that message_headers.php is test ok in ubuntu 16.04 with same kafka server and same librdkafka & php-rdkafka version:

> php message_headers.php
6 messages delivered
%4|1599617807.761|CONFWARN|rdkafka#consumer-2| [thrd:app]: Configuration property dr_msg_cb is a producer property and will be ignored by this consumer instance
Got message: message 0 | Headers: key: value
Got message: message 1 | Headers: key1: value1, key2: value2, key3: value3
Got message: message 2 | Headers: gzencoded: gzdata
Got message: message 3 | Headers: none
Got message: message 4 | Headers: none
Got message: message 5 | Headers: none
@nick-zh
Copy link
Collaborator

nick-zh commented Sep 9, 2020

@douyux thx for the report, so when you:

var_dump($msg->headers);

before the array_walk what is the output?

@douyux
Copy link
Author

douyux commented Sep 9, 2020

var_dump headers before array_walk output is UNKNOWN:0:

code:

$msg = $topic->consume(0, 1000);
if (!$msg || $msg->err === RD_KAFKA_RESP_ERR__PARTITION_EOF) {
    break;
}
if (RD_KAFKA_RESP_ERR_NO_ERROR !== $msg->err) {
    throw new Exception($msg->errstr(), $msg->err);
}
echo 'var_dump:'; var_dump($msg->headers);
$headersString = isset($msg->headers) ? $msg->headers : [];
array_walk($headersString, function(&$value, $key) {
    if ('gzencoded' === $key) {
        $value = gzdecode($value);
    }
    $value = "{$key}: {$value}";
});
if (empty($headersString)) {
    $headersString = "none";
} else {
    $headersString = implode(", ", $headersString);
}
printf("Got message: %s | Headers: %s\n", $msg->payload, $headersString);

result:

[root@xxx test]# /usr/local/php/bin/php message_headers.php 
6 messages delivered
var_dump:NULL
Got message: message 0 | Headers: none
var_dump:UNKNOWN:0
PHP Warning:  array_walk() expects parameter 1 to be array, unknown given in /root/test/message_headers.php on line 82
Got message: message 1 | Headers: none
var_dump:UNKNOWN:0
PHP Warning:  array_walk() expects parameter 1 to be array, unknown given in /root/test/message_headers.php on line 82
Got message: message 2 | Headers: none
var_dump:NULL
Got message: message 3 | Headers: none
var_dump:NULL
Got message: message 4 | Headers: none
var_dump:NULL
Got message: message 5 | Headers: none

@Steveb-p
Copy link
Contributor

Steveb-p commented Sep 9, 2020

@douyux is the PHP version on Ubuntu also PHP 5.6.36?

@nick-zh
Copy link
Collaborator

nick-zh commented Sep 9, 2020

I will try to replicate this when i find some time. Is this happening on PHP7 as well or just PHP5?

@douyux
Copy link
Author

douyux commented Sep 9, 2020

ubuntu php version is: 5.6.40,i'm not sure is will happening on php7 and i think this is nothing to do with php version,maybe the sys & build env cause the issue

@nick-zh nick-zh self-assigned this Sep 9, 2020
@nick-zh nick-zh changed the title Can't parse message header in centos NULL message headers in CentOS7 on PHP5.6 Sep 9, 2020
@douyux
Copy link
Author

douyux commented Sep 10, 2020

this dockerfile can reproduce the isse:

FROM centos:7

RUN yum install -y http://rpms.remirepo.net/enterprise/remi-release-7.rpm \
    && yum install -y yum-utils \
    && yum-config-manager --enable remi-php56 \
    && yum install -y php php-cli php-devel php-pear zlib-devel

RUN yum install -y gcc gcc-c++ automake autoconf libtool make

RUN curl -fsSL 'https://github.com/edenhill/librdkafka/archive/v1.5.0.tar.gz' -o rdkafka.tar.gz \
    && mkdir -p rdkafka \
    && tar -xf rdkafka.tar.gz -C rdkafka --strip-components=1 \
    && rm rdkafka.tar.gz \
    && ( \
        cd rdkafka \
        && ./configure \
        && make \
        && make install \
    ) \
    && rm -r rdkafka

RUN pecl install rdkafka-4.0.3 \
    && echo "extension=rdkafka" >> /etc/php.d/rdkafka.ini

RUN echo $'#!/bin/sh \n\
set -e \n\
exec "$@" \n\
' >> /usr/local/bin/php-entrypoint \
    && chmod +x /usr/local/bin/php-entrypoint

RUN cp /usr/lib64/php/modules/rdkafka.so /usr/lib64/php/modules/rdkafka

ENTRYPOINT ["php-entrypoint"]
CMD ["php", "-a"]

docker run on ubuntu16.04:

docker run -it --rm --name message-headers -v "$PWD":/usr/src/myapp -w /usr/src/myapp qt-centos-php php message_headers.php
6 messages delivered
PHP Warning:  array_walk() expects parameter 1 to be array, unknown given in /usr/src/myapp/message_headers.php on line 82
Got message: message 0 | Headers: none
PHP Warning:  array_walk() expects parameter 1 to be array, unknown given in /usr/src/myapp/message_headers.php on line 82
Got message: message 1 | Headers: none
PHP Warning:  array_walk() expects parameter 1 to be array, unknown given in /usr/src/myapp/message_headers.php on line 82
Got message: message 2 | Headers: none
Got message: message 3 | Headers: none
Got message: message 4 | Headers: none
Got message: message 5 | Headers: none

@nick-zh
Copy link
Collaborator

nick-zh commented Sep 10, 2020

@douyux awesome thx a lot for this, i will check it out ✌️

@nick-zh
Copy link
Collaborator

nick-zh commented Sep 11, 2020

@douyux so what i can report back for now is, that it works for php:7.x under CentOS7. With the deadline coming closer, with us dropping php:5 support, i am not sure how much effort i want to put into this. I will try to check if i can find out why it behaves differently from Ubuntu, but i can't promise anything.

@douyux
Copy link
Author

douyux commented Sep 14, 2020

@nick-zh thanks a lot for the feedback and efforts, it's nice to fix it when you have more free time in the future.

@nick-zh nick-zh removed their assignment Jan 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants