Would you like to train a manim code generation model yourself ?

[Yangget]

Hello macky.

These days I am thinking about how to make a more general path of text -> mainm code -> video.
Since the gpt code and weights are not open, I plan to train a model by myself.
We can work together if you want.

[360macky]

Hello @Yangget! Sorry about delayed answer. Sure, I'd like to hear your idea. Feel free to share it in a new reply for this issue, or on my personal email: hello@marceloarias.com.

[Yangget]

I have sent an email

[waizwafiq]

Hi @Yangget and @360macky ! I'm currently working on a project that involves converting text queries to Manim-specific codes, just like @Yangget mentioned. Then, I stumbled upon this repository. Hahaha.

I would really love to collaborate with you two regarding this project and bring it to life. It would help increase the precision and flexibility of a no-code to low-code programmatic animation platform.

I'll send you guys an email soon so we can discuss on this further! 😄

[Yangget]

Cool! Do you have a discord account? Can I add you？ Waiz Wafiq ***@***.***> 于2023年5月3日周三 13:48写道：

…

Hi @Yangget <https://github.com/Yangget> and @360macky <https://github.com/360macky> ! I'm currently working on a project that involves converting text queries to Manim-specific codes, just like @Yangget <https://github.com/Yangget> mentioned. Then, I stumbled upon this repository. Hahaha. I would really love to collaborate with you two regarding this project and bring it to life. It would help increase the precision and flexibility of a no-code to low-code programmatic animation platform. I'll send you guys an email soon so we can discuss on this further! 😄 — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALA5IGA5RAPJQYKWE3V5FLTXEHWR5ANCNFSM6AAAAAAW534H5U> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[360macky]

Hi @waizwafiq!

That sounds great. It's a good idea that I'm working on.

You can add me to Discord, if it's okay for you: 360macky • Marcelo#7552

[Yangget]

Let's create a Discord Server !

@360macky @waizwafiq

[Puiching-Memory]

Hi!
Can you tell me your latest news? I am very curious about this project

[360macky]

Hi! Can you tell me your latest news? I am very curious about this project

Hi @Puiching-Memory!

We are currently testing an implementation of a flow to generate code inspired by LangGraph:

Our Discord server is small, I invite you to join us. I am updating this experiment continuously these weeks.

[Puiching-Memory]

This link shows that the invitation is invalid, it could be that the link has expired, or there is a problem with my network.
Your idea inspired me. This is my first contact with LangGraph, and I can't help you for the time being.

Recently I have been working on a similar task, hoping to use GPT to drive manim to generate animations. Existing GPT's still make frequent errors. I've heard that there are ways to fine-tune an LLM to turn it into an expert model. I wonder if this method is suitable for the present situation.

If so, what data sets do we need to prepare and which fine-tuning framework should we choose?

[360macky]

This link shows that the invitation is invalid, it could be that the link has expired, or there is a problem with my network. Your idea inspired me. This is my first contact with LangGraph, and I can't help you for the time being.

Recently I have been working on a similar task, hoping to use GPT to drive manim to generate animations. Existing GPT's still make frequent errors. I've heard that there are ways to fine-tune an LLM to turn it into an expert model. I wonder if this method is suitable for the present situation.

If so, what data sets do we need to prepare and which fine-tuning framework should we choose?

Thanks for the heads up on the Discord link. It's updated!

I'm glad to read that the idea inspires you too. I think there are many systems to explore, so I couldn't determine which is best at this point. We started with Zero-Shot Learning, but Fine-Tuning and LangGraph are ideas we explored. Then we will evaluate which is better.

---

For the datasets: We would need datasets that contain the following properties: prompt, code, type.

Basically describe as a human would, a video scene or an image scene for the prompt, categorize it into what would be a video or an image, and the code itself. After this I plan to transform all the code and the prompt (which, in addition to the instruction, will specifically include the type of media to be generated) into the JSONL format that is used for fine tuning.

We have 27 Python files (from Manim Community), usually at least 50+ are preferred (at this point: more is better).

Feel free to propose new ideas, add new files to our recent small dataset, make new things on this workflow or follow the project closely. I'm happy to answer if you have more questions.