Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable require to load Lua modules from within a built LUAZIP application's virtual file system #482

Open
Duckwhale opened this issue Feb 8, 2024 · 1 comment

Comments

@Duckwhale
Copy link
Member

Duckwhale commented Feb 8, 2024

One of the limitations of the current approach to creating standalone apps is that the runtime only requires and loads the entry point (as mentioned in the documentation). It's not difficult to extract files, but I didn't want to port the old import builtin as it has some issues, and luvit's approach of overriding require is even worse. The result is that you can't just require a module that's present in the archive, and any file that was loaded from disk would have to be extracted manually (which makes for a poor UX).

Obviously this feature is quite desirable, but I've put off adding it until I found a simple solution that "just works".

For the first part of the problem (loading Lua scripts), I believe there exists an elegant solution for a custom module loader:

  • By adding a new searcher, custom module loading logic can easily be added to the require system
  • If the runtime registers one that looks into the virtual file system of the LUAZIP archive, it can trivially require any Lua file
  • The only question is whether the searcher should take precedence over reading files from disk
    • Argument for reading from VFS before disk: Makes sure the built app loads the right module if there's name clashes
    • Argument for reading from disk before VFS: Allows overriding behavior without a rebuild, although it may be accidental
    • Due to security concerns and because building is fast anyway, I tend towards reading from the VFS first

I made a quick prototype and there's no issues preventing this to be implemented, things seemed to work as expected:

function evo.run()
	-- ...

	local zipApp = evo.readEmbeddedZipApp()
	if zipApp then
		table.insert(package.searchers, vfs.searcher)
		return vfs.dofile(zipApp, evo.DEFAULT_ENTRY_POINT)
	end

	-- ...
end
-- VFS searcher: Allow optional requires from LUAZIP apps (file system takes priority? (TBD: security risk?))
-- See https://www.lua.org/manual/5.2/manual.html#pdf-package.searchers
function vfs.searcher(moduleName)
	printf("[VFS] Searcher called with moduleName = %s", moduleName)
	if true then
		return function()
			local zipApp = C_FileSystem.ReadFile(uv.exepath())
			printf("[VFS] Searching LUAZIP app %s (%s)", uv.exepath(), string.filesize(#zipApp))
			local filePath = moduleName:gsub("%.", path.separator)
			printf("[VFS] Resolved module name %s to virtual file path %s", moduleName, filePath)
			return vfs.decode(zipApp, filePath)
		end
	end
	-- local errorMessage = "NYI" -- vfs.errorStrings.NOT_YET_IMPLEMENTED
	-- return nil, errorMessage
end
@Duckwhale
Copy link
Member Author

Duckwhale commented Feb 9, 2024

One additional consideration: If a file is successfully loaded from the VFS, and then errors, the stack trace should be useful.

Edit: This is already a given, assuming the use of loadfile with @chunkname. There's just one extra call for the entry point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: It's finally happening! (WIP)
Development

No branches or pull requests

1 participant