Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syscall: syscall_wraps apply to kernel side symbols as well #12293

Open
pussuw opened this issue May 6, 2024 · 3 comments
Open

syscall: syscall_wraps apply to kernel side symbols as well #12293

pussuw opened this issue May 6, 2024 · 3 comments

Comments

@pussuw
Copy link
Contributor

pussuw commented May 6, 2024

System call wraps are used to support system call instrumentation (sched note). This feature is done by the linker, using the --wrap option for each symbol respectively, after the nuttx build is otherwise completed. This is a great feature for monitoring system calls from user space, but unfortunately it applies the same wrapping for kernel side symbols as well, so long as the symbol has the same name as the user side system call does.

One example is the nxsem_xxx APIs. In fact, any API that has the nx- prefix does this.

I'm sure applying the instrumentation for kernel side APIs is not intentional, and very likely I have contributed to this problem in: #11257 where sem_xxx APis were moved to user space.

I'm not sure what to do about this, in my opinion moving the user space related parts of POSIX semaphores (cancel points, errno) into libc was the right call, and I had a plan to continue this with pthreads etc (no time to continue this work for now).

This ticket is basically a placeholder for this issue, as well as an open forum for suggestions on how to fix it.

@xiaoxiang781216
Copy link
Contributor

--wrap is only used in the flat build, for kernel/protected build the hook come from syscall.csv.

@pussuw
Copy link
Contributor Author

pussuw commented May 6, 2024

When SCHED_INSTRUMENTATION_SYSCALL is enabled, --wrap is used to generate wrappers that encapsulate the system call itself into:

int __wrap_syscall_xx(int nr, ...)
{
  sched_note_syscall_enter(int nr, int argc, ...);
  result = __real_syscall_xx();
  sched_note_syscall_leave(int nr, uintptr_t result);
  return result;
}

This is done for every symbol in the syscall table, which includes nxsem_xx et al. The OS calls them directly and as they are now wrapped, they execute the same __wrap_sem_xx call. This is problematic, as it impacts performance as well as clutters the instrumentation with calls that are not system calls from user space (which is what I want to instrument).

@xiaoxiang781216
Copy link
Contributor

Ok, I forget that only the initial version doesn't use --wrap. @YuuichiNakamura switch to --wrap to support the syscall instrumentation in flat build. One possible fix is moving --wrap from kernel bin to userspace libc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants