Setting extent hooks results in error for unitialized arena in default alloc

[antoniofilipovic]

As described in issue, I am trying to use my extent_hooks as a wrapper around the original hooks. When the program is starting, only arena 0 is initialized. If you want to read hooks on other arenas, it is possible to do that, but writing hooks results in error due to the requirement that the arena i needs to be initialized. If arena is not initialized it calls extent_hook alloc function to create arena. My wrapper function calls default hook which results in error:

static void *
extent_alloc_default(extent_hooks_t *extent_hooks, void *new_addr, size_t size,
    size_t alignment, bool *zero, bool *commit, unsigned arena_ind) {
	tsdn_t *tsdn;
	arena_t *arena;

	tsdn = tsdn_fetch();
	arena = arena_get(tsdn, arena_ind, false);
	/*
	 * The arena we're allocating on behalf of must have been initialized
	 * already.
	 */
	assert(arena != NULL);

	return extent_alloc_default_impl(tsdn, arena, new_addr, size,
	    ALIGNMENT_CEILING(alignment, PAGE), zero, commit);
}

It would be better if arena is already initialized and that this part is at least more documented.

My current workaround is reading hooks, writing those same hooks to initialize default arenas, and then setting my hooks:

  uint64_t allocated{0};
  uint64_t sz{sizeof(allocated)};

  sz = sizeof(unsigned);
  unsigned narenas{0};
  int err = mallctl("opt.narenas", (void *)&narenas, &sz, nullptr, 0);

  if (err) {
    return;
  }

  std::cout << narenas << " : n arenas" << std::endl;


  // get original hooks and update alloc
  original_hooks_vec.reserve(narenas);

  for (int i = 0; i < narenas; i++) {
    std::string func_name = "arena." + std::to_string(i) + ".extent_hooks";

    size_t hooks_len = sizeof(old_hooks);
    int err = mallctl(func_name.c_str(), &old_hooks, &hooks_len, nullptr, 0);

    if (err) {
      LOG_FATAL("Error getting hooks for jemalloc arena {}", i);
    }
    original_hooks_vec.emplace_back(old_hooks);

    // Due to the way jemalloc works, we need first to set their hooks
    // which will trigger creating arena, then we can set our custom hook wrappers

    err = mallctl(func_name.c_str(), nullptr, nullptr, &old_hooks, sizeof(old_hooks));

    if (err) {
      LOG_FATAL("Error setting jemalloc hooks for jemalloc arena {}", i);
    }

    err = mallctl(func_name.c_str(), nullptr, nullptr, &new_hooks, sizeof(new_hooks));

    if (err) {
      LOG_FATAL("Error setting custom hooks for jemalloc arena {}", i);
    }
  }

Jemalloc version is 5.2.1

[everschen]

I found the issue, opt.narenas: 64, the huge_arena_ind is 64,
for (int i = 0; i < narenas; i++) {

so you may need to update to
for (int i = 0; i <= narenas; i++) {