Code search does not search critical large files

[rotu]

Select Topic Area

Product Feedback

Body

Some repos depend critically on large source files.
Two examples that have personally been a problem for me:

The core parsing code for the TypeScript compiler: https://github.com/microsoft/TypeScript/blob/main/src/compiler/parser.ts

The canonical ECMAScript language specification https://github.com/tc39/ecma262/blob/main/spec.html

These are omitted from code search omitted from code search with "Your search did not match any code".

https://github.com/search?q=repo%3Amicrosoft%2FTypeScript%20parseImportOrExportSpecifier&type=code
https://github.com/search?q=repo%3Atc39%2Fecma262%20ImageData&type=code

It would vastly increase the usefulness of code search for these repos if large files could be indexed for search, either by default or on an opt-in basis.

[LiteBrite82]

@look answered this post not too long ago: Very large repositories may not be indexed. What is considered a very large repository?

Alternatively, I have had luck using CodeSpaces to run the repo virtually and then searching the term in the VScode web browser (better than pulling it to your local machine):

It took about 2 minutes to build the code space.

Its not ideal, but it works. You can also run git log commands to search through the commit history.

Just be sure to stop the code space when you are done with it - to conserve your hours. Or, delete it if you are just using once.

P.S. This is a regularly requested enhancement; the dev team does monitor this discussion board, though they don't always have the bandwidth to respond.

[LiteBrite82]

The CodeSpace you create is exclusive to the GitHub account you signed into; No one else can see or use it.

[rotu]

Yeah, it's the "Empty files and files over 350 KiB are excluded" limit.

My problem is not that the search isn't doable. It's that:

1. It's not as convenient or discoverable GitHub's own search
2. A negative search result is misleading. It wrongly suggests that a symbol is unused or comes from a different repo.

Good suggestions on using Codespaces!

The best workaround I've found are third-party search engines, e.g.
https://grep.app/search?q=parseImportOrExportSpecifier&filter[repo][0]=microsoft/TypeScript
https://sourcegraph.com/search?q=context:global+repo:%5Egithub%5C.com/microsoft/TypeScript%24+parseImportOrExportSpecifier&patternType=keyword&sm=0

[LiteBrite82]

Thanks for sharing those 3rd party links! I had no clue those existed!