Idea: Remove Lua C-API #236
Comments
|
Hello,
Best Regards |
I would say "drop it". The simpler the better. But I'm not a RaptorJIT or Snabb user, so I have no skin in the game. My 2c. |
|
When you all you have is a hammer, everything looks like a nail. LuaJIT has a pretty good hammer: trace stitching. If the Lua C API is a landmine, then we can trace stitch around it, which makes it less of a performance cliff, and if somebody really needs the performance, they have the FFI. I'm still in favour of dropping the C API, but stitching is an alternative approach, should we reach consensus that we want to keep it. |
|
I am amazed at how complicated this whole discussion is. I am really grateful that people are willing to spend this time teasing out the relationships between the projects. I think this discussion is really important and that establishing clear and respectful boundaries between projects will be worth the effort. This could also help us to understand, over the longer term, which projects and communities should merge and which should follow their own paths. This could be important for the many people maintaining their own internal LuaJIT branches and wondering whether to join with the RaptorJIT community, or stick with LuaJIT community, or form a new splinter community based on more specific goals. |
|
just came back the lost thought into my mind that this is a bit like going against puc lua. btw i realized that writing an ffi connection layer to tekui isnt even a horrible amount of work as i need to replace only a relatively small amount of codes, and not the whole c part, so its not that hard to have compatibility with rj, and i can even keep both versions, so i still wont be cut off from puc world either. so for me its not a really big deal, and thats the only thing that really needs to run in any kinda environment that i colud have. currently i only have this and lfs, but lfs have an ffi variant, and the 2nd most important that would be much to rework is https://github.com/kernelsauce/turbo that is fine without the c api, the 3rd is luakit, that is not fine without it, but its enough to use my pure lua codes in it and communicate with it via the webkit debug api like how the brackets editor does it, and then all my happiness is given :D otoh, im sure that others are around who could have harder tasks... |
|
I'm not a RaptorJIT or Snabb user either, but here are my two cents anyway: The C API is documented in the Language Reference Manual. Alas, this document often does not say what a complying Lua implementation must (should, may) do, but since this API is documented and since it is in spirit of Lua to be easily embeddable, I think one cannot drop this feature and still be an implementation of Lua. After all, I do not agree that it is so hard to maintain the C API. It is already isolated in the code base, so cover it with tests, do not spend time on making it faster (instead, write a corresponding "THIS IS SLOW" warning in the docs motivating your users to use what you think is faster), and forget about this part of the code base for years. On the bright side, you will still remain an implementation of Lua, and once your project will be discovered by someone who needs this feature badly, there will be absolutely no confusion / adoption pain, etc. |
|
The [semi?]official position of the Lua Team, as I understand it, is that LuaJIT is not an implementation of Lua programming language, but rather a separate language, piggy-backing on Lua to get traction. The motivation for this includes things like wildly different recommended ways to write performant code. What is the Lua implementation and what is not? Is RaptorJIT a Lua implementation? Is RaptorJIT a LuaJIT language implementation (whatever that is). Those are important questions, IMHO, and ambiguity here is harmful. (See also Lua in Moscow 2017 proceedings: http://lua.moscow/conf/2017-03-LuaInMoscow/index.html) If RaptorJIT is Lua (I think it is, kind of) and if comformant Lua implementation has to have C API (I think it does not have to --- otherwise it would not be possible to implement Lua in C-less ecosystems, like browser without emscripten), then Lua C API is a hard requirement. Otherwise it is a subject to engineering tradeoff. Fengari, for my money, is clearly a Lua implementation, and it does not have a C API... |
|
Thank you everybody for the $0.02. This is really helpful! @agladysh wrote:
I agree wholeheartedly. If I were the Lua team then I would have mixed feelings about LuaJIT. Sure, imitation is the sincerest form of flattery, but the project could divide the existing community and slow down adoption of new ("unportable") features. LuaJIT would need to somehow benefit the PUC Lua community so that the relationship is symbiotic rather than parasitic. How can we make sure that RaptorJIT treats both PUC Lua and LuaJIT with due respect and doesn't cause unnecessary problems for those projects? |
Define what you are now and what you strive to be very clearly. Live up to that definitions. This, in Lua case, includes defining the rules of how do you define what you are. What is Lua implementation and what is not is a gray area. @igelhaus' pet topic is how to define it better (see also his talk at LiM'17). I think that even helping us to get ahead with this effort would be of no small benefit for the Lua community. ...A practical litmus test: if RJ is Lua, then it should have plans to support 5.3+. RJ may also choose to be "Lua 5.1" frozen in time. Or Lua 5.1 with language extensions. Or LuaJIT clone (whatever that means)... I am not sure what due respect for the LuaJIT would be in this case. Probably the same as above. |
|
I read all answers and I have one stupid question...
If you drop it, what is become impossible to do ? I suppose it is :
I take a look at the thirdparty libraries I use on Lua5.1/5.2/5.3/LuaJIT2.0/2.1 :
If I found a missing feature in RaptorJIT, the only choice is "do it in pure lua code" or "do it over FFI" ? Some Lua's missing features already have different implementation, I think about :
Is the only way to use RaptorJIT will be to run the binary (the interpretor) and doing stuff from it ? I like ljsyscall (and ljsyscall.lfs) but IMHO, it can't cover all the Lua's missing feature ecosystem that is already complex and fragmented. Please, let me know if I'm wrong or missunderstood the discuss and the stakes. |
|
My own two cents (coming from the Terra world):
|
|
@agladysh Seems like it would be very hard for RaptorJIT to commit to being a strict Lua implementation. Just now there are fundamental differences: the way to write efficient code is different and the FFI is the idiomatic way to call C. Then on top of this we don't know how Lua will evolve in the future and whether this would suit us or not -- could be that they make changes that are incompatible with the JIT or with existing RaptorJIT applications. Even if we could, tracking PUC Lua faithfully would break backwards compatibility, and I don't know that this would be the right trade-off to make. So I tend to think that RaptorJIT is its own dialect. How to describe this? If we will be serious about backwards compatibility then a good approximation would be to say where we have started and in which directions we are headed:
Some such statement might be sufficient to manage expectations in the wider Lua-like-languages community? |
|
Just in case anyone is interested, here's the result of grepping for Lua APIs in the Terra source: https://gist.github.com/elliottslaughter/878bd01be7aaf3fc8a1c3f59ed6ad3fe |
To be honest this issue is a bit rhetorical. The C-API is actually used internally in the RaptorJIT codebase so we would not remove the code even if we decided not to support it. The reason it's interesting to discuss is that it's an example of a feature that is normal to use in PUC Lua but should probably never be used in a RaptorJIT program because the FFI is more suitable in pretty much any case. So in a sense we are talking about whether RaptorJIT should support running programs written for a different Lua dialect. In more detail -- The C-API is a way to run C code that gets access to the Lua virtual machine state. On the one hand the C code can call into arbitrary C libraries - sockets, etc - and on the other hand it can directly access the Lua VM state to do things like load parameters from the Lua stack and write back return values. The FFI is different. It does not allow the C code to access the Lua virtual machine state. Lua state is opaque to C code called by the FFI. You don't get a pointer to The difference is very important for the JIT. If you call a C-API function then you have no idea what the state of the virtual machine is anymore. Calling a C-API function automatically invalidates all of the values you have cached in registers, all of the guards you have checked, etc. You cannot really optimize a region of code that contains a C-API call and so the JIT falls back to "stitching" by separately compiling the code on either side of the call. This can really kill performance if you would put a C-API call into an inner loop for example. The FFI is easier. Since C code is not able to see Lua code you know that all the Lua values you have cached in registers are still valid, all your guards about Lua functions/tables/etc are still valid, etc. So you can cheaply insert FFI calls into the middle of optimized Lua code. You only need to make sure that C values are not cached across FFI calls and to save other registers according to the standard platform calling convention. |
|
(Or... I am not quite sure actually how much of the C-API is used internally in the virtual machine itself. The basic calling mechanism is used but I'd have to review the code to know how much API is only consumed by external libraries.) |
|
I have a somewhat extreme suggestion to offer. Make raptorjit its own language that is sufficiently source compatible with lua to allow easy porting of code using it, but not bother with the C API. Add optional type declarations to function parameters, which could help the JIT. Then the C API for raptorjit only exposes a few functions to create a raptor state, to load a string and get a function in the raptor state, to invoke a function, and some convenience routines in the shape of loading raptor code and invoking raptor functions in the VM. LuaJIT has a restricted number of callback functions because it can't rely on the C code to call the function with a reference to the function data. Instead, the call function would accept as it's first argument an opaque reference to a raptor function object, then perform unmarshalling of the arguments exactly like the LJFFI does for callbacks. Callbacks would still be supported and would probably still need the same limitation, but the raptor functions would be able to be freely created and reclaimed. In general, most of the code that uses Lua CAPI calls is in one of two functions: a function callable from lua that just does argument type checking then calls C code with the extracted arguments and then places the result of the C code back into the Lua state, or a function that is callable from C with a C calling convention which just retrieves the corresponding lua function, places the arguments passed by it from C onto the Lua stack and then calls the lua function and unmarshalls the results. Both of these cases could be reduced to autogenerated and jittable code. The third type of operation that is most commonly done using the Lua API would be a small manipulation of the Lua state like adding a field into a table. These cases can be replaced by having small raptor functions that perform the operations. The FFI type marshalling can be extended with types for RaptorFunction and RaptorTable to allow giving an opaque reference to a raptor object to a raptor-aware C function to pass to another raptor function. The reference would almost certainly only be valid in the stack frame, so there would still need to be some concept of a registry and functions to store and fetch from the registry in various ways. This would actually allow writing the Lua CAPI in a combination of Raptor and C by making functions that implement the semantics of the CAPI in raptor. This is certainly less efficient than making the bindings directly, but it would make providing the Lua CAPI be a secondary project that doesn't actually touch the core behaviors and optimizations of the VM and provide an effective upgrade path from the CAPI to raptor bindings. It would be pretty simple to make functions in Lua or Raptor that use a table like the stack and implement operations equivalent to the lua CAPI. If those functions are exposed to C by a common "make this raptor function callable from C" mechanism, then the internals of the VM/JIT no longer needs to regard them as special cases. Basically, kick the responsibility for Lua CAPI to somewhere other than the VM core. Another thing that might be useful to this discussion is the project https://github.com/Neopallium/LuaNativeObjects used in https://github.com/libgit2/luagit2 which allows autogenerating both LUACAPI and LJFFI bindings for a C library from a simple lua description of it. This tool will allow making C modules that are compatible with both standard lua and raptorjit if the ffi stays compatible with LuaJIT but the CAPI breaks compatibility with Lua. |
|
@lukego Below are my replies, more from the exploring the "what is Lua?" point of view. NB: I ask some dumb questions there, they are not entirely rhetorical.
This is probably true, especially since RJ is descended from LuaJIT.
TBH, this would be a difference (of different impact) for any different Lua implementation.
This is not quite a language level difference, isn't it? At least for me, FFI is more a kind of a built-in module. The same as e.g. Fengari would provide to interact with JS in browser.
This is important and valid consideration, and it makes you fail the litmus test. [I wonder, the way Lua development is governed now (that is the way I like, don't get me wrong), wouldn't most serious Lua implementations have this problem?]
IMHO, over the years, either you break it, or you end up with "JavaScript" mess of a language.
What about implicit and incomplete 5.2 support LuaJIT has? (Yet another harmful ambiguity, IMHO.)
Is this an important information?
May these be language level features? E.g. new, non-Lua syntax? [Interestingly, this is where the border lies for me personally. If a Lua-descended project introduces a custom syntax, it instantly becomes not interesting for me — it breaks "The Lua way". Again, 2c, I'm not your target audience anyway.]
Is this too an important information when defining a language / dialect?
This item seems to be more about platform, and not language. Does not harm to define that, but I would separate language definitions from platform definitions. Same probably holds true for some of the above items.
It would. |
Maybe this is a feature not a bug? Would the PUC Team prefer if other people didn't make separate-but-compatible implementations? Maybe these other implementations are a distraction from what would otherwise be a cosy single-implementation language?
Yes. For me the syntax, the standard libraries, the performance characteristics of the runtime system, etc, are all one whole. I would consider adding a new syntactic construct as a minor change compared with (say) changing the performance of the garbage collector. If a new syntax construct comes along I can ignore it but if the GC latencies increase I might have to rewrite my whole application. So I'd say that we have opposite views. For me a language specification is not Truth but just part of the documentation for a compiler. (Just acknowledging this and not arguing about which view is better -- that has been done to death on the internet for decades.)
I see it as a spectrum with JavaScript/C++ at one extreme and Lua/Scheme at the other. Lots of languages in the middle evolve gradually over time and provide a healthy balance between new features and backwards compatibility. Erlang, for example, which is also a language where the characteristics of the BEAM virtual machine are at least as important as the language definition itself.
Aside: I find it very encouraging that you are able to decide this. If we are successfully communicating who this project is for then it should also be clear who it is not for. If everybody were excited about the future of RaptorJIT that would be a troubling sign of poor communication :). |
|
Thanks @elliottslaughter for sharing these details. This is an interesting use case that I hadn't been conscious of before. |
I like this sentiment. RaptorJIT need not aim to be strictly compatible with Lua nor LuaJIT but we might aim to make porting easy. That way people can spend some time running LuaJIT and RaptorJIT side-by-side while they decide which is more suitable for their project, or might maintain support for both if RaptorJIT doesn't currently support all their target platforms. This should not take too much time and effort or else RaptorJIT will be too hard to evaluate and adopt. The C-API has some value in this sense because maybe for some reason you are using it, for example because there is a specific library that you would prefer to reuse rather than port over to FFI, and the JIT compilation issues aren't a problem for your specific usage. Or maybe you would want to port the library to FFI over the longer term but you don't want that to block your testing of RaptorJIT. |
I have no problem with this point of view, it is reasonable, just coming from a different set of tradeoffs than mine. This is why I said that we'll have to define how to define what Lua (or RaptorJIT) is. Please think about including the above near to the bullet point list of what is the RaptorJIT, it would help. |
|
I don't understand if the C-API you might drop includes the possibility to call Lua code from C. |
|
The idea is you write your program in RaptorJIT, not to extend a C program.
…On Tue, 19 Feb 2019, 10:05 Henri D ***@***.*** wrote:
I don't understand if the C-API you might drop includes the possibility to
call Lua code from C.
Could raptor still be used to extend a C software? (and not only about
calling C from Lua through FFI).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#236 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABbx2wZgAdWWLYLaGgwb7QubIzSTuOBsks5vO8xkgaJpZM4bAYRV>
.
|
|
There's also one important use of Lua C API - C++ binding libraries, which use Lua C API and therefore are compatible with both PUC Lua and LuaJIT. By dropping Lua C API you're making it almost impossible to create support for RaptorJIT in, let's say, sol2 Maybe @ThePhD can add something about his experiences with using Lua C API when making sol2. |
|
Related: How does OpenResty integrate with nginx? C-API, FFI, or both? Anybody care to explain? |
|
Given that
https://github.com/openresty/lua-nginx-module/blob/master/src/api/ngx_http_lua_api.h
includes lua.h, it's probably using the C API.
…On Tue, 19 Feb 2019, 13:05 Luke Gorrie ***@***.*** wrote:
Related: How does OpenResty integrate with nginx? C-API, FFI, or both?
Anybody care to explain?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#236 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABbx2xU0sbwCou9sUboEF9e0NrMl0KCNks5vO_aegaJpZM4bAYRV>
.
|
|
Also luajit.h : https://github.com/openresty/lua-nginx-module/blob/master/src/ngx_http_lua_common.h |
|
I seem to have been called into this discussion! In my experience, most of the way higher-level languages consume and expose scripts like Lua in C++ is to use a wrapper around the C API. However, there's a lot of discontent with the C API in regards to performance and other things. I can only speak truly to C++, but the pain points for C++ and C developers generally come from:
In general, Lua also has this problem for continued development:
Those are the general grievances, but let's focus on what not having the C API means: Removing the C API is essentially removing the go-to way to talk to Lua from not just C, but from at least a dozen other languages in which C is the binding glue that helps them talk without doing crazy things. I suspect you could write RaptorJIT bindings specifically, but then people would need to write specifically to you. Also keep in mind that a large portion of debugging and hooking for Lua development comes from the Hook C API. Removing that is nuking tools that want to use RaptorJIT as the backend across the board. I can't imagine that amount of "vendor" (not that you're specifically selling this) lock-in is fairly undesirable. Overall, I consider not having the Lua C API to be a backwards step for the community in general, but maybe you'll be able to bring some benefit by not having the C API that can make good on some chunk of your community (less development time spent, etc.?). Side notes:
So, yeah. Initial analysis says that if you're trying to write something for a specific use case or company, do whatever you want, including kill the C API (that's what literally dozens of privately hacked Lua implementations did). If you're trying to increase RaptorJIT adoption and make it a backend that's easy to transfer to, make it a Lua 5.3+ C API (other people can write compat wrappers). If you're willing to break everything, then I suggest the 2 biggest things you think about doing is adding nil support for arrays, giving arrays an O(1) size, and adding language syntax for it (so that it is a superset of Lua; do not fundamentally change the syntax of other language constructs unless you're okay with throwing portability of backends and codebases out the window). Lua 5.4 is not looking incredibly ambitious (they got rid of LUA_NIL_IN_TABLES). They sometimes do share the direction they're going in with work1/work2 releases and betas, but usually the development is closed so tracking the specification and keeping up with it will always be a lagging affair for both the Language and the Library, Lua and C API. (Albeit, I believe they recently have spent quite a bit of time committing larger portions of work more frequently to a Lua-owned github, wherein you could track much more closely what is changing and what is not?) ... And now I'm rambling. But that's all the stuff I know about this space. Good luck with RaptorJIT!!! |
|
Thanks for chiming in @ThePhD! This is a great summary of how things look from your perspective. My impression is that you are exactly the target user for PUC Lua. You are writing applications in C++, you want to embed Lua as a scripting language, you are willing to do the necessary work to migrate to new versions. You would not be in the target audience for RaptorJIT then. This is for people who want to use Lua instead of C++. Crossing the Lua/C boundary is primarily about Lua calling into C (or assembler) to handle exceptional situations like code that needs to use special instructions like SIMD, AES, CRC32 that aren't supported in Lua. Or to call an external library or system call. In these situations it is important to use the FFI instead of the C-API for the sake of efficiency. If people don't realize this, e.g. because they are leaning too heavily on their experience with PUC Lua, it will cause them frustration.
On the one hand we don't want to break things: we are more about backwards compatibility than PUC Lua does and this is what prevents us from tracking Lua development unless/until it becomes more stable. On the other hand making a clean break from any expectation of PUC Lua compatibility is tempting. I don't want to distract people like you from focusing on the latest PUC Lua features, I don't want to give the false impression that idiomatic PUC Lua programming style is suitable for RaptorJIT, and I sure as heck don't want to spend time making the JIT optimize code that was written for a different VM. My feeling is that each Lua dialect needs a mostly separate software ecosystem. I want authors of popular Lua libraries to say "we support PUC Lua and not RaptorJIT" rather than "we support both PUC Lua and effing jeffing I don't want to onboard PUC Lua users to RaptorJIT. They already have a perfectly fine Lua to enjoy. I want to onboard Python/perl/Ruby/JavaScript/Lisp/Smalltalk/etc users who need more performance for their project and don't want to switch to a static language like C. Goal is not to divide the current Lua community but to create a new related one. Currently the first sentence of our README says:
This is supposed to attract people who want to write a whole application in Lua instead of C/C++/D/Rust and to repel people who want an embedded scripting language or a drop-in replacement for PUC Lua. Following this discussion I think we should at least replace "Lua implementation" with "Lua dialect" to emphasise that we have forked the language. Beyond that I'm not sure?
Thank you! and for the concrete feature ideas too. |
|
So, I was rereading the LuaJIT quad-colour garbage collector, and it was mentioned that because LuaJIT has to interact with the C API, it cannot move data. This rules out techniques like generational garbage collection. While it's a good idea to let allocation sinking do its thing, allocations on the heap at the moment cripple an application (I got a 4x speedup by avoiding GC in my hot path). Mike's design appears to not have been as capable as he planned, according to this paper, so there could be notable worst-case speedup if a generational collector was used. |
|
@ZirconiumX Could continue the GC discussion by reviving issue #62 "New garbage collector?" Sounds like you have found some new relevant information. |
|
hi there again! :) as im still not familiar with these enough maybe im asking some dumb stuffs, but what about os level multithreading and what about multiple lua states with any kinda interaction between them in the same app? are those still possible? i can think about using shared memory for whatever intention like this could still do the trick, but i think uve got more knowledge about these... if u can say that anything like these are possible, then fine, but i dont even have the mental model for knowing what im really asking about, just i believe those are nice possibilities, like the ability of using multiple cpu cores... so thx if u can make me calm and my visions about these fields any cleaner! :D btw some interesting more or less related brainstorming for the internals in case if anyone missed it: |
|
@szbnwer Multi-thread is something I've been thinking about quite a bit, because my workload relies on that to be fast. At the moment, as far as I can tell, large parts of the LuaJIT VM are not thread-safe, ranging from the internal Honestly in the distant future I'd like an M:N threading system, based on coroutines or whatever. |
No, this does not seem justified to me at this time. I'd welcome a PR to remove any custom allocator code in favour of calling the standard routines. |
|
hi! :) ive never played with pthreads and the like, but only with simple event loops, so ive got only a not so strong view about threads, but the two benefits i think that actually important are the usage of multi core cpu's and that its not necessary to wait for yielding, so these can bring performance and real time stuffs can work finely. these are important abilities that i wanna undermine in the future. btw multiple rj instances could do the trick for these with some interoperability, but i think that can be easily solved. otoh stuffs like jit compilation on an another cpu core could give some extra horsepower that always sounds good. whats wrong with dlmalloc? ive read its codes mostly, but that was my 1st allocator implementation that ive seen, so im not about standing up for it, but im interested in ur point of view, while stated that im not totally unfamiliar with it, so thats an ease for u to say whatever about it for me :D my best guess is that its just an another piece of optimalization, but if i read ZirconiumX right, then its against multithreading. - am i right? what else? other caveats? more mess in the codebase? how much beneficial is it related to the standard allocators? thx 4 anything! :) |
|
On Sun, 3 Mar 2019, 19:00 szbnwer, ***@***.***> wrote:
hi! :)
ive never played with pthreads and the like, but only with simple event
loops, so ive got only a not so strong view about threads, but the two
benefits i think that actually important are the usage of multi core cpu's
Yeah, this is what I would like, too.
and that its not necessary to wait for yielding,
This is known as preemptive multitasking, and things like Erlang use it. I
personally think it's not going to play well with the JIT, because what
happens when you preempt in the middle of a trace recording? If anything
this should be left to the OS, rather than us.
so these can bring performance and real time stuffs can work finely.
Cooperative threading handles
these things reasonably well if you yield often enough.
these are important abilities that i wanna undermine in the future. btw
multiple rj instances could do the trick for these with some
interoperability,
OS inter-process communication is a *nightmare*. Finding an IPC concept
portable between OSes is very difficult.
but i think that can be easily solved. otoh stuffs like jit compilation on
an another cpu core could give some extra horsepower that always sounds
good.
This doesn't really work in the current architecture of RJ, because the JIT
trace recorder hooks the interpreter to record opcodes.
whats wrong with dlmalloc? ive read its codes mostly, but that was my 1st
allocator implementation that ive seen, so im not about standing up for it,
but im interested in ur point of view, while stated that im not totally
unfamiliar with it, so thats an ease for u to say whatever about it for me
:D my best guess is that its just an another piece of optimalization, but
if i read ZirconiumX right, then its against multithreading.
It's both. It's 1,400 lines of code which is hopefully not necessary, and
it's a hindrance to multi-threading.
- am i right? what else? other caveats? more mess in the codebase? how
much beneficial is it related to the standard allocators?
At the moment we don't know; @lukego was going to run some benchmarks.
… thx 4 anything! :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#236 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABbx243Ogy21g66ncfVcfu9uqjgEDJXfks5vTBu9gaJpZM4bAYRV>
.
|
previously i didnt see how hard that could be, now i can imagine it :D
IF... but i think there can be cases when its not suitable to rely upon this, even if the possibility to make it well is given.
ive got no experience with anything better than using the fs, but is that the case if i care only about linux? (currently only hurd looks like an interesting stuff, and mayb embedded device os'es can be in my interest other than these, but no closed source stuffs like win and mac, and not even bsd.) i could think about shared memory and process io with named pipes could make such things faster than putting serialized data into files, but i dont have currently a usecase where fs is not suitable...
isnt it possible to just reshape it while keep its benefits in case if it turns out that its very efficient? and thx for all the info! :) |
|
If we cared about malloc performance, why not use jemalloc?
Or, even better, RaptorJIT already has an allocator: the GC.
…On Mon, 4 Mar 2019, 00:40 szbnwer, ***@***.***> wrote:
because what happens when you preempt in the middle of a trace recording?
If anything this should be left to the OS, rather than us.
This doesn't really work in the current architecture of RJ, because the
JIT trace recorder hooks the interpreter to record opcodes.
previously i didnt see how hard that could be, now i can imagine it :D
Cooperative threading handles these things reasonably well if you yield
often enough.
IF... but i think there can be cases when its not suitable to rely upon
this, even if the possibility to make it well is given.
OS inter-process communication is a *nightmare*. Finding an IPC concept
portable between OSes is very difficult.
ive got no experience with anything better than using the fs, but is that
the case if i care only about linux? (currently only hurd looks like an
interesting stuff, and mayb embedded device os'es can be in my interest
other than these, but no closed source stuffs like win and mac, and not
even bsd.) i could think about shared memory and process io with named
pipes could make such things faster than putting serialized data into
files, but i dont have currently a usecase where fs is not suitable...
It's both. It's 1,400 lines of code which is hopefully not necessary, and
it's a hindrance to multi-threading.
isnt it possible to just reshape it while keep its benefits in case if it
turns out that its very efficient?
and thx for all the info! :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#236 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABbx25Xfxh94JOaV4n_jU-mt2sMTSdcGks5vTGuCgaJpZM4bAYRV>
.
|
|
I wonder if the Lua C-API could be a very valuable reflection tool. The other day I did a hack that displays a Lua stack trace when Snabb encounters a segmentation fault. The Lua C-API was a nice too for doing that sort of thing. It would maybe be even better if it was more "read-only" (you have to use the Lua stack to operate the API). So this is not to say I think we should keep the API as is, but rather that an API with similar functionality to inspect the heap and stack memory of a RaptorJIT process post mortem or snapshot could be really useful? I have read about people calling the Lua C-API from GDB to do debugging etc... cool stuff imho. |
|
NB: Yesterday at Lua in Moscow 2019 I asked Roberto a question in an interview: if someone would remove Lua C API from a Lua implementation, would it still be a Lua implementation? My interpretation of the answer is: the Lua language design tradeoffs were so heavily influenced by the existence of the Lua C API over the years, that if one removes the API, one may just as well change the language design to pick a new, now-better, set of tradeoffs. I will post a link to the interview video once it will be available. I will see Roberto on Wednesday again, and can continue the discussion, if you all be interested and will help me with the feedback. |
|
I think it's entirely possible to build an introspection API which is
better suited to the internals of RaptorJIT, as opposed to using the
sledgehammer C API.
I was talking to @kyren on Reddit about Luster, which is a Rust Lua VM.
Compatibility with the C API is an explicit non-goal, and I am inclined to
agree with the reasoning that it is very difficult to abstract safely over.
At what point do we say we have consensus about this? I could go through
and summarise the conversation so far if that would help.
…On Mon, 4 Mar 2019, 12:28 Alexander Gladysh, ***@***.***> wrote:
NB: Yesterday at Lua in Moscow 2019 <https://lua.moscow> I asked Roberto
a question in an interview: if someone would remove Lua C API from a Lua
implementation, would it still be Lua.
My interpretation of the answer is: the Lua language design tradeoffs were
so heavily influenced by the existence of the Lua C API over the years,
that if one removes the API, one may just as well change the language
design to pick a new, now-better, set of tradeoffs.
I will post a link to the interview video once it will be available.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#236 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABbx2wziQ8jse6oFqWwH8ZKudAwklO_tks5vTRFvgaJpZM4bAYRV>
.
|
Here is a radical idea - something we should consider and discuss seriously - not necessarily something we will do:
How about if we remove the Lua C-API from RaptorJIT? Idea from #235 (comment).
This would be one way to draw a line in the sand between Lua and RaptorJIT. The downside is that it breaks compatibility between various Lua libraries and RaptorJIT. The upside is that it frees everybody from being "caught in the middle" and focused on the lowest-common-denominator that is portable between Lua/LuaJIT/RaptorJIT.
Meanwhile many handy utilities written in pure Lua could still be easily used/ported with RaptorJIT since the language remains reasonably compatible.
Case in favour? Case against?
Related: discussion on the Terra repo terralang/std#4 where quasi-compatibility between Lua and LuaJIT is causing headaches for application developers who have to decide between Lua, LuaJIT, or lowest-common-denominator portable subset. This is no way to live! I'd much prefer to give users something they can fully embrace (or not.)
The text was updated successfully, but these errors were encountered: