Cleanstrtab #472

rohoog · 2023-03-02T21:01:58Z

I have created the change that adds cleaning of the dynstr. All unreferenced symbols are removed if --clean-strtab option is given and the section is shrunk. It could solve issues #453 and #449.

However I found that the binary actually grows by about 4KB. I think this is due to this note section normalizing. I'm not sure why this is done and why this should be needed. Why not leave all sections before the one(s) that are modified alone?

brenoguim

Overall, it seems there are many changes in this PR, some needed for the functionality and some cosmetic. This makes it a bit hard to review with confidence.

Also, adding a test is pretty important, so it would go a long way in helping the PR to be accepted.

src/patchelf.cc

brenoguim · 2023-03-05T15:42:26Z

src/patchelf.cc

+
+#include <argz.h>
+
+class mymap : public map<symstr, forward_list<symref>, Comparator> {


Perhaps one would want a better name for this class

What would you suggest? symmap or such?

brenoguim · 2023-03-05T15:42:46Z

src/patchelf.cc

+    auto strTab = (char *)fileContents->data() + rdi(shdrDynStr.sh_offset);
+    int verneednum = 0;
+    int verdefnum = 0;
+    //mkref(strTab, rdi(shdrDynStr.sh_size)); // new strInfo()


cleanup old code

weird, this line is gone already here.... commit "remove some leftover comment-out lines " should have removed it.

brenoguim · 2023-03-05T15:43:33Z

src/patchelf.cc

    changed = true;
 }

 template<ElfFileParams>
-void ElfFile<ElfFileParamNames>::clearSymbolVersions(const std::set<std::string> & syms)
+void ElfFile<ElfFileParamNames()>::clearSymbolVersions(const std::set<std::string> & syms)


It would be nicer if these changes weren't made in this PR, or at least in a different commit

I'll (try to) split it in cosmetic commit and functional commit.

brenoguim · 2023-03-05T15:43:58Z

src/patchelf.cc

 }

+struct symstr {


Perhaps one could use a std::string_view for this?

maybe... This looks most like the original C-implementation I did. However, I need none of the 'burden' of string_view.

looked into string_view and the constructor string_view(char s) can be useful to replace the argz_ functions too. Good tip,

brenoguim · 2023-03-05T15:48:35Z

src/patchelf.cc

-static uint64_t roundUp(uint64_t n, uint64_t m)
-{
-    return ((n - 1) / m + 1) * m;
-}
-


is this change related?

no, part of the 'cosmetic' changes.

brenoguim · 2023-03-05T15:49:08Z

src/patchelf.cc

@@ -600,7 +614,7 @@ void ElfFile<ElfFileParamNames>::writeReplacedSections(Elf_Off & curOff,
        const std::string & sectionName = i.first;
        Elf_Shdr & shdr = findSectionHeader(sectionName);
        if (rdi(shdr.sh_type) != SHT_NOBITS)
-            memset(fileContents->data() + rdi(shdr.sh_offset), 'X', rdi(shdr.sh_size));
+            memset(fileContents->data() + rdi(shdr.sh_offset), 'Y', rdi(shdr.sh_size));


is this necessary?

left-over from my debugging

brenoguim · 2023-03-05T15:49:37Z

src/patchelf.cc

-
-        curOff += roundUp(i->second.size(), sectionAlignment);
+        debug("curoff %d, size %d, align %d\n", curOff, i->second.size(), sectionAlignment);
+        curOff += roundUp((Elf_Off)i->second.size(), sectionAlignment);
+        debug("-> next curoff %d\n", curOff);


What's the purpose of this change?

debugging left-over

brenoguim · 2023-03-05T15:50:28Z

src/patchelf.cc

@@ -1514,7 +1529,7 @@ void ElfFile<ElfFileParamNames>::modifyRPath(RPathOp op,
    size_t rpathSize = 0;
    if (rpath) {
        rpathSize = strlen(rpath);
-        memset(rpath, 'X', rpathSize);
+        memset(rpath, 0, rpathSize);


Is this change necessary?

debugging left-over

brenoguim · 2023-03-05T15:51:12Z

src/patchelf.cc

+    if (!changed) return;
+	auto newsize = rdi(shdrDynamic.sh_size) - sizeof(Elf_Dyn) * (dyn - last);
+	wri(shdrDynamic.sh_size, newsize);
+	dynamic.resize(newsize);
+	debug("new .dynamic size %d\n", rdi(shdrDynamic.sh_size));
+	replaceSection(".dynamic", dynamic);



Misleading indentation

yeah, this used to be if (changed) {...

brenoguim · 2023-03-06T19:19:20Z

I also forgot to say. You might want to enable the CI in your github fork so that you can make sure all tests are passing.

brenoguim · 2023-03-06T19:22:48Z

We are going to need to decide between my implementation and yours.
Mine has the disadvantage of undoing the sharing, but I intend to implement sharing detection on it, which would then create sharing for the modifications.
Yours persist the sharing, but would not add sharing. I suspect adding sharing detection on my implementation would be more fiting the algorithm, while in yours it would need a more significant rewrite.
What do you think about it?

rohoog · 2023-03-06T21:18:52Z

Sorry, I didn't look into your implementation. Mine was a C++ learning experience for me. Do with it as you please.
I addressed your review comments, except the split of the initial big commit and the adding of the test case (if you did a test-case, then that would probably work for my code too). Please have a look.
Initially I also had the idea of adding the symbol sharing, until I found that the gnu binutils already does that, so I didn't bother. Since the 'patching' is about removing DT_NEEDED, I don't think there will be new opportunities for sym sharing.

I do find the growing of the binary disturbing. Also the number of program headers increases by the 'rewriteSections' method. Should I make a new issue for that?

brenoguim · 2023-03-07T00:01:47Z

Since the 'patching' is about removing DT_NEEDED, I don't think there will be new opportunities for sym sharing.

Patchelf has many operations that will grow the dynstr table. For example, changing RPATH, or renaming symbols. So redoing the sharing could improve in those cases.

I do find the growing of the binary disturbing. Also the number of program headers increases by the 'rewriteSections' method. Should I make a new issue for that?

There is definitely room for improvement indeed. Patchelf will run a generic algorithm to change the place of modified sections, and to do that it needs to add more loads.
But if you know that you did not grow any section, you can just tune the algorithm to avoid calling rewriteSections.

r-hoogenboom · 2023-03-07T08:24:27Z

My changes are all about shrinking sections, not growing. Do you mean that I don't need to call rewriteSections after shrinking? What else should I call to 'propagate' the shrinking into the section headers and program headers and yield a smaller binary?

brenoguim · 2023-03-07T09:52:08Z

I understand that. I just meant that it would be useful to redo the sharing because some options in patchelf add new strings.

If you just set changed=true without rewriting the sections, the binary will have exactly the same size. At least it won't grow. Patchelf will just update the changed bytes in the file.

If you want to actually shrink, I guess this needs to be implemented.

It's complicated because if you want to keep the section in the same place as it was before and make the file smaller, that would mean all other sections would be loaded in an earlier address space, which breaks the binary.
If you want to move the new smaller section to another place in the virtual space, you need to add a new program header table entry. And maybe there isn't room for that, forcing you to displace the next section into another location. But to displace sections, you might have to normalize note sections... So you get to the issue existing today.

r-hoogenboom · 2023-03-07T10:48:50Z

HMM, I expected that exactly this was the responsibility of rewriteSections. If it can deal with grown sections, why wouldn't it be able to deal with shrunk sections.

Note sections are just arbitrary 'advisory' non-critical things (like which assembler assembled the binary, which linker linked it, etc), right? Do they really refer to absolute locations of data in the binary?

I think patchelf is growing to be a full-blown linker (and un-linker) and by that, one that supports many architectures and bit-nesses. I think patchelf should leave stuff alone which it can, try to leave as much as possible to the linker that originally built the binary. But that's just my opinion.

Ronald added 3 commits February 28, 2023 08:48

add cleaning of dynstr

8ba25a5

remove some leftover comment-out lines

9ab525e

Merge branch 'master' into cleanstrtab

fe3bafa

rohoog mentioned this pull request Mar 2, 2023

Implementing --clean-strtab #461

Open

brenoguim reviewed Mar 5, 2023

View reviewed changes

Ronald added 4 commits March 6, 2023 21:04

address some cleanup suggestions

9264068

get rid of a warning about casting away const

da45be2

rename mymap to symmap

a8ffdc6

use string_view instead of my own struct and get rid of argz dependency

2b8a29e

Merge branch 'master' into cleanstrtab

fe9b475

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cleanstrtab #472

Cleanstrtab #472

rohoog commented Mar 2, 2023

brenoguim left a comment •

edited

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

rohoog Mar 6, 2023

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

rohoog Mar 6, 2023

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

brenoguim Mar 5, 2023

rohoog Mar 6, 2023

brenoguim commented Mar 6, 2023

brenoguim commented Mar 6, 2023

rohoog commented Mar 6, 2023

brenoguim commented Mar 7, 2023

r-hoogenboom commented Mar 7, 2023

brenoguim commented Mar 7, 2023

r-hoogenboom commented Mar 7, 2023


		#include <argz.h>

		class mymap : public map<symstr, forward_list<symref>, Comparator> {

Cleanstrtab #472

Are you sure you want to change the base?

Cleanstrtab #472

Conversation

rohoog commented Mar 2, 2023

brenoguim left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brenoguim commented Mar 6, 2023

brenoguim commented Mar 6, 2023

rohoog commented Mar 6, 2023

brenoguim commented Mar 7, 2023

r-hoogenboom commented Mar 7, 2023

brenoguim commented Mar 7, 2023

r-hoogenboom commented Mar 7, 2023

brenoguim left a comment •

edited