Windows VSCode error: Error: EMFILE: too many open files

[xrkk]

Does this issue occur when all extensions are disabled?: Yes/No

• VS Code Version: 1.56.2
• OS Version: Windows Enterprise LTSC

Steps to Reproduce:

In a workspace with too many files, when trying to search symbol with Ctrl+T, nothing happends, no symbols show up, and log window:

[2021-05-19 20:25:07.653] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\z_工具与库\mongo\资源.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\z_工具与库\mongo\资源.md'
	at _handleError (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:87:156611)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:124073)
	at async Promise.all (index 8188)
	at async s.getAllMarkdownDocuments (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:122844)
	at async l.populateSymbolCache (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:125142)
	at async l.provideWorkspaceSymbols (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:124545)

If in linux, there is a way to fix this: https://code.visualstudio.com/docs/setup/linux#_visual-studio-code-is-unable-to-watch-for-file-changes-in-this-large-workspace-error-enospc
But on Windows, how can I do this?

[vscodebot]

(Experimental duplicate detection)
Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

• TS Server fatal error: Cannot read property 'lastIndexOf' of undefined (#120563)
• TS Server fatal error: Cannot read property 'lastIndexOf' of undefined (#123061)
• TS Server fatal error: Cannot read property 'lastIndexOf' of undefined (#123026)
• Crash on TypeScript semantic highlighting (#120875)

[mjbvz]

@jrieken Assigning you since I believe the markdown extension is just calling vscode.workspace.fs.readFile here:

vscode/extensions/markdown-language-features/src/features/workspaceSymbolProvider.ts

Line 82 in d5f4e11

private async getMarkdownDocument(resource: vscode.Uri): Promise<SkinnyTextDocument | undefined> {

[xrkk]

@jrieken By the way: this happens when all extentions disabled.

[bpasero]

@xrkk can you clarify what OS Windows Enterprise LTSC is?

[xrkk]

@xrkk can you clarify what OS Windows Enterprise LTSC is?

Version: Windows 10 企业版 LTSC
Build Number: 1809
OS Internal Version: 17763.1935

[bpasero]

I don't know what the file service can do here, after all you are hammering the disk with requests, this code is really dangerous:

vscode/extensions/markdown-language-features/src/features/workspaceSymbolProvider.ts

Lines 29 to 33 in d5f4e11

	async getAllMarkdownDocuments() {
	const resources = await vscode.workspace.findFiles('**/*.md', '**/node_modules/**');
	const docs = await Promise.all(resources.map(doc => this.getMarkdownDocument(doc)));
	return docs.filter(doc => !!doc) as SkinnyTextDocument[];
	}

Because it is potentially unbounded and you run all these things in parallel without limits.

I would suggest to revisit the choice, at the minimum implement some debouncing.

[bpasero]

Oh hello: isaacs/node-graceful-fs#160

Looks like my adoption of fs.promises disables graceful fs...

I still think markdown should not be doing this and will open a separate issue.

[xrkk]

I tried latest Insider version, and problem still exists.

[2021-05-31 10:18:44.618] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\zz_doc\docs\re\IDA Pro\scripting\ida-minsc\examples.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\zz_doc\docs\re\IDA Pro\scripting\ida-minsc\examples.md'
	at _handleError (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:85:157610)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125034)
	at async Promise.all (index 8177)
	at async s.getAllMarkdownDocuments (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:123805)
	at async l.populateSymbolCache (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:126103)
	at async l.provideWorkspaceSymbols (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125506)
[2021-05-31 10:21:04.518] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\z_Python\库\命令行\riposte.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\z_Python\库\命令行\riposte.md'
	at _handleError (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:85:157610)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125034)
	at async Promise.all (index 8136)
	at async s.getAllMarkdownDocuments (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:123805)
	at async l.populateSymbolCache (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:126103)
	at async l.provideWorkspaceSymbols (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125506)
[2021-05-31 10:21:19.612] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\反分析&反检测&反xx\.NET&C#\工具\2019.使用运行时代码编译将在另一个程序集中隐藏程序集.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\反分析&反检测&反xx\.NET&C#\工具\2019.使用运行时代码编译将在另一个程序集中隐藏程序集.md'
	at _handleError (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:85:157610)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125034)
	at async Promise.all (index 7514)
	at async s.getAllMarkdownDocuments (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:123805)
	at async l.populateSymbolCache (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:126103)
	at async l.provideWorkspaceSymbols (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125506)
[2021-05-31 10:21:29.579] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\0_论文\反-分析\z_dup\2015.Ropinjector-Using return oriented programming for polymorphism and antivirus evasion.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\0_论文\反-分析\z_dup\2015.Ropinjector-Using return oriented programming for polymorphism and antivirus evasion.md'
	at _handleError (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:85:157610)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125034)
	at async Promise.all (index 8188)
	at async s.getAllMarkdownDocuments (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:123805)
	at async l.populateSymbolCache (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:126103)
	at async l.provideWorkspaceSymbols (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125506)
[2021-05-31 10:21:41.097] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\C&C\DGA\文章\2019.从恶意软件Nymaim看DGA.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\C&C\DGA\文章\2019.从恶意软件Nymaim看DGA.md'
	at _handleError (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:85:157610)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125034)
	at async Promise.all (index 7942)
	at async s.getAllMarkdownDocuments (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:123805)
	at async l.populateSymbolCache (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:126103)
	at async l.provideWorkspaceSymbols (c:\Users\Admin\Downloads\VSCode-win32-x64-1.57.0-insider\resources\app\extensions\markdown-language-features\dist\extension.js:1:125506)

Version Info:

Version: 1.57.0-insider
Commit: 541a5d4fa70ab3095f23b28b88914e1f6a8f65c4
Date: 2021-05-28T09:50:57.063Z
Electron: 12.0.9
Chrome: 89.0.4389.128
Node.js: 14.16.0
V8: 8.9.255.25-electron.0
OS: Windows_NT x64 10.0.17763

I copied whole folder to Ubuntu Virtual Machine and tried an older version of VSCode, and this problem does not occur.

Version: 1.54.3
Commit: 2b9aebd5354a3629c3aba0a5f5df49f43d6689f8
Date: 2021-03-15T10:55:24.277Z
Electron: 11.3.0
Chrome: 87.0.4280.141
Node.js: 12.18.3
V8: 8.7.220.31-electron.0
OS: Linux x64 5.8.0-45-generic

And, the folder has more than 15K note files.

[xrkk]

@bpasero I think this issue should be re-opened.

[bpasero]

Thanks a lot, indeed not fixed. But I am too afraid to tackle this in May because the solution is a bit more involved.

[xrkk]

I understand.

My note folder has ~17K files, including: ~9K markdown files, ~5K files in .git folder and ~3K png image files.

Using VSCode Ctrl+T to navigate through tags is quite helpful, is there anything I can do to make it work instead of waiting for issue being fixed?

[xrkk]

Now I open the folder with WSL-Windows, and it works!

[bpasero]

via 708951c

[xrkk]

@bpasero Not fixed in latest version.

版本: 1.57.0 (system setup)
提交: b4c1bd0
日期: 2021-06-09T17:18:42.354Z
Electron: 12.0.9
Chrome: 89.0.4389.128
Node.js: 14.16.0
V8: 8.9.255.25-electron.0
OS: Windows_NT x64 10.0.17763

[2021-06-11 06:31:31.165] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\APT\文章\2020.全球APT2020年中报告（奇安信）.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\APT\文章\2020.全球APT2020年中报告（奇安信）.md'
	at _handleError (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:84:155978)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:124957)
	at async Promise.all (index 8056)
	at async s.getAllMarkdownDocuments (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:123732)
	at async l.populateSymbolCache (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:126026)
	at async l.provideWorkspaceSymbols (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:125429)
[2021-06-11 06:32:53.860] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\0_论文\反-分析\Anti-Environment\2021.Sandbox Detection Using Hardware Side Channels.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\0_论文\反-分析\Anti-Environment\2021.Sandbox Detection Using Hardware Side Channels.md'
	at _handleError (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:84:155978)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:124957)
	at async Promise.all (index 8188)
	at async s.getAllMarkdownDocuments (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:123732)
	at async l.populateSymbolCache (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:126026)
	at async l.provideWorkspaceSymbols (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:125429)
[2021-06-11 06:32:58.893] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\0_论文\反-分析\z_Anti-Anti\2020.(HybridEmu)Hybrid emulation for bypassing anti-reversing techniques and analyzing malware.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\0_论文\反-分析\z_Anti-Anti\2020.(HybridEmu)Hybrid emulation for bypassing anti-reversing techniques and analyzing malware.md'
	at _handleError (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:84:155978)
	at runMicrotasks (<anonymous>)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:124957)
	at async Promise.all (index 8189)
	at async s.getAllMarkdownDocuments (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:123732)
	at async l.populateSymbolCache (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:126026)
	at async l.provideWorkspaceSymbols (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:125429)
[2021-06-11 06:33:03.363] [renderer1] [error] Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\0_论文\反-分析\z_z\2018.AntiVirus software evasion-an evaluation of the AV Evasion tools.md': Unknown (FileSystemError) (FileSystemError): Error: EMFILE: too many open files, open 'e:\_\note\恶意代码\0_论文\反-分析\z_z\2018.AntiVirus software evasion-an evaluation of the AV Evasion tools.md'
	at _handleError (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:84:155978)
	at runMicrotasks (<anonymous>)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async s.getMarkdownDocument (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:124957)
	at async Promise.all (index 8188)
	at async s.getAllMarkdownDocuments (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:123732)
	at async l.populateSymbolCache (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:126026)
	at async l.provideWorkspaceSymbols (c:\Program Files\Microsoft VS Code\resources\app\extensions\markdown-language-features\dist\extension.js:1:125429)

I think it might not be the probelm of Promise.all, since it works in Remote-WSL-Window.

[bpasero]

@xrkk yeah you will need to switch to our insiders version to benefit from the fix. It should be out in a moment today.

You can give our preview releases a try from: https://code.visualstudio.com/insiders/

[xrkk]

Yes. Fixed in insider version. Thank you!