Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#2183: LDMS: add stream for phase data #2184

Draft
wants to merge 11 commits into
base: develop
Choose a base branch
from
Draft

#2183: LDMS: add stream for phase data #2184

wants to merge 11 commits into from

Conversation

lifflander
Copy link
Collaborator

Fixes #2183

@lifflander lifflander linked an issue Aug 10, 2023 that may be closed by this pull request
Create LDMS stream publish for phase data #2183
Open
@github-actions
Copy link

github-actions bot commented Aug 11, 2023 •
edited

Pipelines results

PR tests (gcc-12, ubuntu, mpich)

Build for 1c98fba (2023-11-09 17:49:25 UTC)

FAILED: src/CMakeFiles/vt.dir/Unity/unity_20_cxx.cxx.o 
/usr/bin/ccache /usr/lib/ccache/g++ -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DVT_NO_COLOR_ENABLED -I/vt/lib/CLI -I/build/vt/release -I/vt/src -I/vt/lib/json/include -I/vt/lib/brotli/c/include -I/vt/lib/libfort/lib -isystem /ovis/LDMS_install/include -isystem /vt/lib/fmt/include -isystem /vt/lib/EngFormat-Cpp/include -isystem /build/checkpoint/install/include -O3 -DNDEBUG -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -std=c++17 -MD -MT src/CMakeFiles/vt.dir/Unity/unity_20_cxx.cxx.o -MF src/CMakeFiles/vt.dir/Unity/unity_20_cxx.cxx.o.d -o src/CMakeFiles/vt.dir/Unity/unity_20_cxx.cxx.o -c /build/vt/src/CMakeFiles/vt.dir/Unity/unity_20_cxx.cxx
In file included from /vt/src/vt/vrt/collection/manager.impl.h:69,
                 from /vt/src/vt/vrt/collection/manager.h:1773,
                 from /vt/src/vt/scheduler/scheduler.cc:49,
                 from /build/vt/src/CMakeFiles/vt.dir/Unity/unity_20_cxx.cxx:5:
/vt/src/vt/vrt/collection/balance/node_lb_data.h:267:24: error: 'nlohman' has not been declared
  267 |   void writeJSONToLDMS(nlohman::json& j);
      |                        ^~~~~~~
/vt/src/vt/vrt/collection/balance/node_lb_data.h:267:37: error: expected ',' or '...' before '&' token
  267 |   void writeJSONToLDMS(nlohman::json& j);
      |                                     ^
FAILED: examples/collection/CMakeFiles/migrate_collection.dir/Unity/unity_0_cxx.cxx.o 
/usr/bin/ccache /usr/lib/ccache/g++ -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DVT_NO_COLOR_ENABLED -I/vt/lib/CLI -I/vt/lib/json/include -I/vt/lib/brotli/c/include -I/vt/lib/libfort/lib -I/build/vt/release -I/vt/src -isystem /ovis/LDMS_install/include -isystem /vt/lib/fmt/include -isystem /vt/lib/EngFormat-Cpp/include -isystem /build/checkpoint/install/include -O3 -DNDEBUG -fdiagnostics-color=always -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -std=c++17 -MD -MT examples/collection/CMakeFiles/migrate_collection.dir/Unity/unity_0_cxx.cxx.o -MF examples/collection/CMakeFiles/migrate_collection.dir/Unity/unity_0_cxx.cxx.o.d -o examples/collection/CMakeFiles/migrate_collection.dir/Unity/unity_0_cxx.cxx.o -c /build/vt/examples/collection/CMakeFiles/migrate_collection.dir/Unity/unity_0_cxx.cxx
In file included from /vt/src/vt/vrt/collection/manager.impl.h:69,
                 from /vt/src/vt/vrt/collection/manager.h:1773,
                 from /vt/src/vt/vrt/collection/collection_headers.h:49,
                 from /vt/src/vt/transport.h:69,
                 from /vt/examples/collection/migrate_collection.cc:44,
                 from /build/vt/examples/collection/CMakeFiles/migrate_collection.dir/Unity/unity_0_cxx.cxx:3:
/vt/src/vt/vrt/collection/balance/node_lb_data.h:267:24: error: 'nlohman' has not been declared
  2%0D%0A%0D%0A%0D%0A ==> And there is more. Read log. <==

Build log

@JacobDomagala JacobDomagala force-pushed the 2183-create-ldma-stream-publish-for-phase-data branch from 365191b to cacd48a Compare September 30, 2023 13:16
@JacobDomagala JacobDomagala self-assigned this Oct 2, 2023
@JacobDomagala JacobDomagala force-pushed the 2183-create-ldma-stream-publish-for-phase-data branch 2 times, most recently from 8baa8fc to 7b49e32 Compare October 3, 2023 13:02
@JacobDomagala
Copy link
Contributor

Code used for reading the data:

#include <ldms/ldms.h>
#include <ldms/ldmsd_stream.h>
#include <ovis_util/util.h>

int stream_handler(ldmsd_stream_client_t c, void *ctxt,
                    ldmsd_stream_type_t stream_type,
                    const char *data, size_t data_len,
                    json_entity_t entity)
{
    // Process the received data
    printf("%.*s\n", (int)data_len, data);

    return 0;
}

int main()
{
    ldms_init(256);
    ldms_t ldms = ldms_xprt_new_with_auth("sock", "none", NULL);
    int rc = ldms_xprt_connect_by_name(ldms, "localhost", "10444", NULL, NULL);
    if(!rc){
      printf("Error code %d \n", rc);
    }
    ldmsd_stream_subscribe("LB_data", stream_handler, NULL);
    while (1){
         // Keep running to continue receiving data
    }
    ldms_xprt_put(ldms);
    return 0;
}

Config file:
export HOSTNAME=localhost
export COMPONENT_ID=1

load name=hello_sampler
config name=hello_sampler producer=${HOSTANAME} instance=${HOSTNAME}/hello_sampler component_id=${COMPONENT_ID} stream=LB_data

Script to launch the LDMS deamon:

#!/bin/bash

TOP=/ovis/LDMS_install
export LD_LIBRARY_PATH=$TOP/lib/:$TOP/lib:$LD_LIBRARY_PATH
export LDMSD_PLUGIN_LIBPATH=$TOP/lib/ovis-ldms
export ZAP_LIBPATH=$TOP/lib/ovis-ldms
export PATH=$TOP/sbin:$TOP/bin:$PATH
export PYTHONPATH=$TOP/lib/python2.7/site-packages


ldmsd -x sock:10444 -c /ldmsd.conf -l /tmp/demo_ldmsd_log -v DEBUG -r $(pwd)/ldmsd.pid

@JacobDomagala JacobDomagala force-pushed the 2183-create-ldma-stream-publish-for-phase-data branch 5 times, most recently from caca0d1 to 1939ddb Compare October 10, 2023 16:18
@lifflander lifflander force-pushed the 2183-create-ldma-stream-publish-for-phase-data branch from 1939ddb to 1c98fba Compare November 9, 2023 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@JacobDomagala JacobDomagala

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Create LDMS stream publish for phase data
2 participants
@lifflander @JacobDomagala