Skip to content

Automate the analysis of GitHub repositories for LLMs with RepoToTextForLLMs. Fetch READMEs, structure, and non-binary files efficiently. Outputs include analysis prompts to aid in comprehensive repo evaluation

Notifications You must be signed in to change notification settings

Doriandarko/RepoToTextForLLMs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 

Repository files navigation

RepoToTextForLLMs

hero

Automates the analysis of GitHub repositories specifically tailored for usage with large context LLMs. This Python script efficiently fetches README files, repository structure, and non-binary file contents. Additionally, it provides structured outputs complete with pre-formatted prompts to guide further analysis of the repository's content.

Features

  • README Retrieval: Automatically extracts the content of README.md to provide an initial insight into the repository.
  • Structured Repository Traversal: Maps out the repository's structure through an iterative traversal method, ensuring thorough coverage without the limitations of recursion.
  • Selective Content Extraction: Retrieves text contents from files, intelligently skipping over binary files to streamline the analysis process.

Prerequisites

To use RepoToTextForLLMs, you'll need:

  • Python installed on your system.
  • The github Python package.
  • A GitHub Personal Access Token configured as an environment variable (GITHUB_TOKEN).

Getting Started

  1. Ensure Python and the required package (PyGithub) are installed:
pip install PyGithub tqdm
  1. Set your GitHub Personal Access Token as an environment variable:
GITHUB_TOKEN = os.getenv('GITHUB_TOKEN', 'YOUR TOKEN HERE')

How to Use

  1. Place the script in your desired directory.
  2. Execute the script in your terminal:
python repototxt.py
  1. Enter the GitHub repository URL when prompted. The script will process the repository and output its findings, including the README, structure, and file contents (excluding binary files), accompanied by analysis prompts.

Contributing

Contributions to RepoToTextForLLMs are welcomed. Whether it's through submitting pull requests, reporting issues, or suggesting improvements, your input helps make this tool better for everyone.

License

This project is licensed under the MIT License.

About

Automate the analysis of GitHub repositories for LLMs with RepoToTextForLLMs. Fetch READMEs, structure, and non-binary files efficiently. Outputs include analysis prompts to aid in comprehensive repo evaluation

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages