Add library support to Luacontrollers

[cheapie]

This allows mods to provide their own libraries that can be accessed from within a Luacontroller, for example to make working with advanced digilines peripherals somewhat easier.
Libraries can be added to the mesecon.luacontroller_libraries table, and then the code running in the Luacontroller can use require() to request one. require() will return nil if the library is not present.

As some examples of where this could be useful:

• One of my mods (digistuff) provides a "touchscreen" that can display various GUIs, but controlling it is rather difficult. I have written a library to do it (TSLib) but at the moment the only way to use the library is to copy/paste it into the beginning of the LuaC program. This change would allow digistuff to supply the library itself, and LuaC programs could simply require("TSLib") to obtain a copy.
• On servers that use lightweight interrupts, a mod could provide various helper functions to implement IIDs in another way, or possibly provide an alternative to the whole interrupt() mechanism altogether.
• A server owner may desire to make certain game-related functions (such as minetest.get_craft_result(), minetest.get_server_status(), or possibly even the entire "minetest" table in singleplayer) available within Luacontrollers. This would allow that to be done without editing the mesecons_luacontroller mod itself.

[Desour]

👍 for adding something like this, and for using require.
But I don't see the need to change the envs of the funcs.

[Desour]

Why the new variable? newfunc is still the same function as v.
You are setting the env of v. This might be useful for functions that use functions from the luacontroller env. But in general I think this is at least rather inconvenient. If you want to make the whole minetest table accessible, like you suggested in the title comment, you will have to wrap nearly every function because otherwise it will not work, or crash because it's not a lua function.

[cheapie]

I suppose I misunderstood the function passing behavior in Lua here - after reading up on that some, I see I could clean this up a little. But the setfenv() is still needed (I can just do it on v) in order for the main purpose of these libraries - interacting with LuaC capabilities - to work. Shoving the entire "minetest" table in there was more of just sort of a wild suggestion and not really an intended use case.

[Desour]

You could make this more general by having library getter functions in mesecon.luacontroller_libraries:

Suggested change

	local function get_require(env)
	return function(name)
	if mesecon.luacontroller_libraries[name] then
	return mesecon.tablecopy_change_env(mesecon.luacontroller_libraries[name],env)
	end
	end
	end
	local function get_require(env, pos)
	return function(name)
	if mesecon.luacontroller_libraries[name] then
	return mesecon.luacontroller_libraries[name](env, vector.new(pos))
	end
	end
	end

Note that I've also added pos here. I think there was somewhen the request to be able to add some sort upgrades to the luacontroller, like a sensor node that is placed beneath the luacontroller, having the pos allows this.

Such a getter function could still change the env of their functions if it wants to. And it has to ensure there are no security flaws, but that's the case anyway, a simpler api without the deepcopy-with-env-change might make this even easier.

[Desour]

It would be nice to have this mentioned somewhere, eg. in the code comment at the top of the file.

[cheapie]

for adding something like this, and for using require.
But I don't see the need to change the envs of the funcs.

The envs need to be changed, otherwise the library functions are unable to access normal Luacontroller functionality (digiline_send(), interrupt(), event, pin, port, etc.)

[Desour]

Yes I know. But if you use functions in mesecon.luacontroller_libraries, as suggested, you can generate the libraries depending on the env (and so access the env stuff via setfenv of your function or using the env as upvalue in your library functions).

[cheapie]

Hmm... how about if I was to allow libraries to be registered either as a table or a function - if they're a table then it would do what ti does now, or if it's a function then it would be called as you suggested. This way you could have all sorts of advanced stuff like you suggested, whereas simple libraries like the ones I was more after don't need to specially handle the environment themselves.

[Desour]

¯\_(ツ)_/¯
The current way of changing the env modifies your given functions every time a luacontroller executes its code, and imo is just weird.
Consider these examples:

local function f1(c)
	digiline_send(c, "bla")
end
local function f2(c)
	f1(c.."2")
end
local function f3(p)
	return minetest.get_node(p)
end
local get_node = minetest.get_node
local function f4(p)
	return get_node(p)
end
local incl_both = false
mesecon.luacontroller_libraries["foo"] = {
	f2 = f2,
	f1 = incl_both and f1 or nil,
	f3 = f3,
	f4 = f4,
}

In the luacontroller f2 won't work because f1 can't access digiline_send, but if you set incl_both to true it will work. Also, f3 doesn't work, but f4 does.
I consider such things too confusing to be the default.

With the more general approach, it's more explicit, but not really more complex imo:

mesecon.luacontroller_libraries["foo"] = function(env, pos)
	local function f1(c)
		env.digiline_send(c, "bla")
	end
	local function f2(c)
		f1(c.."2")
	end
	local function f3(p)
		return minetest.get_node(p)
	end
	local get_node = minetest.get_node
	local function f4(p)
		return get_node(p)
	end
	local incl_both = false
	return {
		f2 = f2,
		f1 = incl_both and f1 or nil,
		f3 = f3,
		f4 = f4,
	}
end

You could allow both ways, but I wouldn't encourage using the one that sets the envs.

Edit: Also, the standard luacontroller env functions use the normal env, so I could also reason with consistency.

Edit2: The libraries you think of probably do something that could be done with the normal luacontroller functions (=> require instead of copy). Such things could be done by local lib = loadfile(path); mesecon.luacontroller_libraries["foo"] = function(env, pos) return setfenv(lib)() end;, then you even have your loacontroller code in a separate file where you can make a normal module and do all the stuff you can do in the luacontroller. (But beware of require-cycles that are not handled by the current require implementation.)

Btw. this PR fixes #428.

[cheapie]

I will admit that the current way is a little strange in regards to how the environment behaves, but it's the cleanest way I could come up with for libraries that just aim to do things that the LuaC could already do, just without copy/pasting a bunch - like the one for the touchscreen that just gives a bunch of functions that eventually call digiline_send(), or in #428 where the goal is just to supply (copies of) a table without any functions at all.

Both of these are doable in the way you suggest, but with additional requirements (the mod registering it would need to supply a getter function).

Give me a few minutes here and I'll see about coming up with a (hopefully) relatively clean way to just handle both.

[BuckarooBanzay]

Is this still relevant/active?

I think this feature will open up a whole new ecosystem of pluggable libraries and i'm looking forward to that 👍

Can i help somehow here? Does this need further testing or documentation?

[cheapie]

Is this still relevant/active?

I don't know - I pushed that last set of changes after the previous message and never heard anything about it after that.

[TurkeyMcMac]

The string metatable sandbox should probably be mentioned in the documentation. Furthermore, require should exit the string metatable sandbox while loading libraries. It might also be good to store loaded libraries and return them if they are required again, as this would match the behavior of the "real" require.

[BuckarooBanzay]

fyi: this has been forked and implemented here: https://github.com/mt-mods/mesecons not sure if this is still relevant/feasible to merge here

[TurkeyMcMac]

To elaborate on the string metatable point: Say the library function is called at

mesecons/mesecons_luacontroller/init.lua

Line 473 in 91e3b13

return mesecon.luacontroller_libraries[name](env, pos)

and it in turn calls another function. This function may not expect to be run inside the string metatable sandbox. If it tries to call s:find with a pattern as an argument, for example, it will crash unexpectedly.

[numberZero]

👍 for the idea.
👎 for changing function environments. That will cause myriads of mysterious bugs.

Actually there are 2 kinds of libraries IMO:

1. Controller-local, that only need to run “inside” the LuaC. These can be provided as code chunks and be run in the LuaC environment entirely.
2. Controller-boundary, like digilines. These are, by definition, crossing a security boundary so simplistic things are non-solutions there. Time limit alone can wreak havoc on a perfect-looking solution.

[numberZero]

Repetitive mesecon.luacontroller_libraries[name]