Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: improve storage tool checks, disable default allocator check, fix mwcsys_executil #289

Merged
merged 7 commits into from
May 17, 2024

Conversation

678098
Copy link
Collaborator

@678098 678098 commented May 14, 2024

Storage tool:

  • remove manual memory management
  • add preconditions checks
  • list all member fields on construction
  • make the code robust to NULL allocator

@678098 678098 requested a review from a team as a code owner May 14, 2024 16:17
Signed-off-by: Evgeny Malygin <emalygin@bloomberg.net>
Signed-off-by: Evgeny Malygin <emalygin@bloomberg.net>
Signed-off-by: Evgeny Malygin <emalygin@bloomberg.net>
Signed-off-by: Evgeny Malygin <emalygin@bloomberg.net>
Signed-off-by: Evgeny Malygin <emalygin@bloomberg.net>
@678098 678098 changed the title [WIP]Fix: AIX tests Fix: improve storage tool checks, disable default allocator check, fix mwcsys_executil May 17, 2024
@@ -130,7 +130,7 @@ static void test2_executeSystemFailure()

PVV("Testing abnormal exit of the command");
rc = mwcsys::ExecUtil::execute(&output,
"python -c 'import os,signal; "
"python3 -c 'import os,signal; "
Copy link
Collaborator Author

@678098 678098 May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot use implicit python2 here, python3 alias might work on most frequently known systems
We still might be in trouble if we don't have python3 alias on a system but have only a specific version like python3.11. For me, it's worth trying to test
Ideally, the test should use a tool with more rigid (stable) name, but it requires more time to fix

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this test. If it fails, we can use use python3.11 etc, but I'd rather not tie ourselves to a specific Python version if we can help it.

Signed-off-by: Evgeny Malygin <emalygin@bloomberg.net>
s_ignoreCheckDefAlloc = true;
// Disable default allocator check for this test until we can debug
// it on AIX/Solaris
#endif
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what code do allocations wrongly, it is difficult to find without means to build and test it on AIX/Solaris. I reviewed the storage tool and improved many things related to allocations, but maybe we should just disable this check until we debug it on the needed platforms

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me. It's at least better to do this test by default on Linux, which will give us at least some early indications with GH Actions.

Signed-off-by: Evgeny Malygin <emalygin@bloomberg.net>
Copy link
Collaborator

@pniedzielski pniedzielski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻 as long as CI passes

s_ignoreCheckDefAlloc = true;
// Disable default allocator check for this test until we can debug
// it on AIX/Solaris
#endif
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me. It's at least better to do this test by default on Linux, which will give us at least some early indications with GH Actions.

@@ -130,7 +130,7 @@ static void test2_executeSystemFailure()

PVV("Testing abnormal exit of the command");
rc = mwcsys::ExecUtil::execute(&output,
"python -c 'import os,signal; "
"python3 -c 'import os,signal; "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this test. If it fails, we can use use python3.11 etc, but I'd rather not tie ourselves to a specific Python version if we can help it.

@@ -102,8 +102,8 @@ class JournalFile {
// Number of records.
MappedFileDescriptor d_mfd;
// Mapped file descriptor.
char* d_mem_p;
// Pointer to allocated memory for journal file.
bsl::vector<char> d_mem;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vector is fine (exception safe!). I'm surprised there isn't a clear BDE component for this use case, but I couldn't find one with a quick search.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe allocating data buffers via vector is a common practice for safe (not a joke, but still sounds funny) C++ programming, just as using smart pointers and other RAII things

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the past days, we used levmar lib for optimizations in C++ code, with API like this:

int dlevmar_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata)

And we were really happy to use vector buffers on top of this

@678098 678098 merged commit ef0508c into bloomberg:main May 17, 2024
14 checks passed
@678098 678098 deleted the 240514_fix_AIX_build branch May 17, 2024 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants