By skyforbes 
Hi everyone, I just wanted to share a project I did over the last weekend.
I’m no ML engineer or having any relevant background in AI, just have been toying with the idea of training an LLM myself for a while.
Most of my previous training attempts did not yield and meaningful result, but I’m still managed to learned a thing or two. And this time, I decided to give it a try again.
The niche language I picked to train the LLM (Qwen2.5-coder-7b) was a less popular text-to-diagram language called Pintora. Since most open source models did not have any knowledge about this language, it’s a fun project to try.
Long story short, I planned to train this for free on Google Colab, but ended up renting a 48GB A40 for a naive mistake, and doing a lot of the training pipeline myself (in a much smaller scale), from creating the dataset, cleaning them up, to do two phases training: Continued Pretraining and then Instruction Finetune, to teach the model how to either generate diagrams from scratch and editing existing diagrams.
In the end, I’m quite happy with the result, although it’s not great, the model was able to generate syntactically correct code, the diagrams are showing up. I did a quick evaluation to confirm how accurate (in terms of of compile-able diagrams) that the model can generate, out of 1000 examples, only about 140 are failing, that’s about 86% accuracy.
Both the model (safetensors, gguf, full and quantized) are available on HF if you are interested. I also did a write up to document the process, I think it might be helpful to share so I can learn from all of your feedback!
Blog post: https://huy.rocks/everyday/12-01-2025-ai-teaching-an-llm-a-niche-diagraming-language
Model:
- https://huggingface.co/huytd189/pintora-coder-7b
- https://huggingface.co/huytd189/pintora-coder-7b-gguf
ataset: