Other LORA models

[philwee]

Would it be possible to support other LORA adapters?

For example, I've finetuned llama on alpaca + dolly (https://huggingface.co/couchpotato888/dolpaca_gpt4_13b_1e_adapter/tree/main) but I can't seem to use it on your Colab (it tells me it's unsupported) - it would be really nice if I could use your interface with my finetune.

Thanks for the great work on it btw, the interface looks really nice!

[deep-diver]

sure thing! @philwee

It shows "unsupported" since I have not tested the model. Please let me know where the model is(apparently you pointed it), well working cases(or how to structure the prompts). I assume you followed the same structure as in the original Alpaca dataset, yes?

[deep-diver]

I manage prompt interfaces for different models in different repositories(for example plain Alpaca), if you think this one exactly matches to yours, please let me know

[philwee]

It's based on basically plain alpaca, so it should work with the same stuff as alpaca - happy to test it once support is given (i have both a 7b and 13b adapter). Thank you for your help!

[deep-diver]

Absolutely! Will let you know when I integrate them :) pretty soon

[deep-diver]

hey @philwee ,

can you share the GenerationConfig that you used?
When I say "hello", it gives me such as (too) long responses which is weird, and I can even easily find some weird words or ending tokens.

Generated text on "hello"

Hello, what is your name? | What can I do for you today.J it depends a lot on how specific are you with detail! Tell me your need.
Thank you | Please follow these steps (step-1 to 5) Danny van Emden – Slowing Things Down For Health and Longevity
By Alan McCubbin|2019-07-30T14:56:58+11:00May 8th, 2019|Cycling Tips Podcast Sponsors, Interviews/General Chat, Race Reports|0 Comments
DJV has been around at an elite level in cycling Australia now since the start of this century so we really wanted him back to share some of his knowledge which he will take us through over a number of different questions below… He’s competed from junior world champions in track to winning Giro stages among other highlight wins… All whilst still holding up his Masters title as one of our top exponents across all levels… Not forgetting to also have fun along the way… We look forward to hearing more stories down the track. Big thanks again DJV… Hope that helps inspire you out there listening to hear about someone like himself who slowed everything down only after doing such fast speeds beforehand but to then realising that actually slowing down made better faster than when he raced superfast!!! Have patience if it takes time… Don’t worry too much if it doesn’t happen at first either… just listen and think later about why Djv thinks things are done certain ways!!! Cheers ACM :) [Audio Clip Below]: https://www.cyclingtips.com.au/episode-player/12929/-dannyvanemdenslatingthingforhealthandlongeivity.mp3 Subscribe now… Get access early
This podcast is brought to […]
Gran Fondo Campagnolo Gold Coast Q & A Featuring Pauline Ferrand Prévot AKA The Prevodouille
As you would be aware, Cyclingnews reported at the end of last month French mountain biker Marion Rambert broke her femur while tackling “a technical descending section” during racing and she didn’t come anywhere near a barrier or get run off by another rider as often assumed. On Monday the International Cycling Union issued UCI rule changes stating there would no longer be any pro cross-country races held prior to the Rio Olympic Games beginning in August due the dangers the sport was causing its athlete pool according to multiple sources I had heard speaking openly previously who know their history in depth regarding what the past Olympic medallists have expressed they had experienced themselves. My initial gut reaction without having enough recent info from reliable medical professionals within cyclocross however were concerns whether it perhaps may turn into similar experiences seen recently within XC MTB where the most skilled individuals have had to sit out big events even while injured. If you add into consideration further reports on social media from cyclists not affiliated directly related […]
Newcastle to Wagga Week 3 Day Review Feeling Fantastic!
By Alan McCubbin|2019-05-19T03:45:21+11:00November 16th, 2014|Training Log- Diary|Comments Off on Newcastle to Wagga Week 3 Day Review Feeling Fantastic!
We had arrived here yesterday quite sore but happy knowing if we could just eat well sleep deep tonight feel nice tomorrow night it’ll all slowly ease and then just focus on riding feeling good next week instead!! Yasmin came round for dinner which helped my spirits greatly… She said we needed to ride out tonight because we need less time training before the event rather stay fit!! She seemed keen I guess haha :). Her cousin Sammi from Tasmania turned up shortly after that to give us support driving tomo which meant she also stayed… So yay double excited!! So many new faces coming into Team Vampetta life it’s truly something special right ! Tonights team includes Jaxx, Jesse and Dave who I rode a couple events with earlier in year plus Al who had just finished Kona World Championships… There are a few new ones too for example Nina is joining up late in event once we pass Canberra for final leg up as we head north- west inland route from Dubbo… Plus lots and lots of amazingly beautiful volunteers helping in anyway possible - it really is mind boggling seeing and experiencing!.. This group really knows that food means family haha everyone talking over meal - and being thankful of it makes it easy tough not taking that personally when so many people show

[philwee]

hi @deep-diver - I tested it using tloen/alpaca-lora's generate.py, which defaults to using following prompt:

{
    "description": "Template used by Alpaca-LoRA.",
    "prompt_input": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Input:\n{input}\n\n### Response:\n",
    "prompt_no_input": "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:\n",
    "response_split": "### Response:"    
}

More info can be found here: alpaca-lora prompts

Right now, I just tested it - when I give it the input of "hello", I get "Hello! How can I help you today?", which seems alright - see below:

Please let me know if this works, otherwise, happy to try to help.

[deep-diver]

I see, the Generation config is so different. let me check

[deep-diver]

I think I can make this model works with just a simple case (1, 1, 1 instruction, input, and response). I have found that it is hard to inject past conversation history in the input, so I decided not to use that template in this project.

However, in your model's case, it looks it doesn't work if I don't follow the standard template format. hm... gotta think about this any further to make your model to work better.

[philwee]

I think it was intended to be used in an alpaca like style anyway (1 input, 1 output) - and the data that it was trained on was like that anyway. For now, I think that simple case would still be nice to have! :)

I can also try to fine tune another version that focuses on a more conversational style - I'll update you once I've worked on it.

[deep-diver]

For sure! How would you format dataset to reflect conversational style?

[deep-diver]

probably I will include your model as a single turn conversation since I plan to add StackLLaMA as well

[philwee]

So I initially finetuned it using data in the format instruction-input-output (as is needed by alpaca-lora)

For example:

[
    {
      "instruction": "Give three tips for staying healthy.",
      "input": "",
      "output": "1. Eat a balanced and nutritious diet: Make sure your meals are inclusive of a variety of fruits and vegetables, lean protein, whole grains, and healthy fats. This helps to provide your body with the essential nutrients to function at its best and can help prevent chronic diseases.\n\n2. Engage in regular physical activity: Exercise is crucial for maintaining strong bones, muscles, and cardiovascular health. Aim for at least 150 minutes of moderate aerobic exercise or 75 minutes of vigorous exercise each week.\n\n3. Get enough sleep: Getting enough quality sleep is crucial for physical and mental well-being. It helps to regulate mood, improve cognitive function, and supports healthy growth and immune function. Aim for 7-9 hours of sleep each night."
    }
]

A more conversation like format would involve data that looks something like this (note the delimiters [|Human|] and [|AI|])

[
       {
          "topic": "Give three tips for staying healthy.", 
          "input": "The conversation between human and AI assistant.\n[|Human|] Give three tips for staying healthy.\n[|AI|] 1.Eat a balanced diet and make sure to include plenty of fruits and vegetables. \n2. Exercise regularly to keep your body active and strong. \n3. Get enough sleep and maintain a consistent sleep schedule.\n[|Human|] "
        }
]

(in my testing, the training with the conversation style-like data appears to result in better chat output, especially more than 1q+1a, than the instruction-input-output format)

[philwee]

probably I will include your model as a single turn conversation since I plan to add StackLLaMA as well

cool, thank you! stack llama is a nice approach too (in that it uses RLHF)

[philwee]

Update: I've finetuned another model with data in a more conversational format: https://huggingface.co/couchpotato888/baize7bdollylora
(here it is in adapter format, lmk if you need it in hf format)

This one should work better for chat use, ex: