phi3

[francescoagati]

phi3 have a good support for function calling but dont work with phidata

[jacobweiss2305]

@francescoagati , we have intergration with phi3. What issues are you having?

We are super active on discord:
https://discord.com/invite/4MtYHHrgA8

Happy to help you get this figured out

[francescoagati]

in many case using ollama with phi3 dont work give empty response

[francescoagati]

with function calling tools sometimes dont call tools

[francescoagati]

but now i have that using pydantic work gookd

from phi.assistant import Assistant
from phi.llm.ollama import Ollama
from phi.tools.duckduckgo import DuckDuckGo
from phi.tools.wikipedia import WikipediaTools
from phi.tools.website import WebsiteTools

#import pydantic
from pydantic import BaseModel, Field


class Article(BaseModel):
    """
      A Pydantic model for an article
    """
    title: str = Field(..., title="Title of the article")
    description: str = Field(..., title="Description of the article")
    body: str = Field(..., title="Body of the article")


assistant = Assistant(
    tools=[WikipediaTools()], 
    show_tool_calls=True,
    debug_mode=True,
    output_model=Article,
    llm=Ollama(
        model="phi3",
        ))


assistant.print_response("a long article with title description and body about buddhism?", markdown=False)

[jacobweiss2305]

@francescoagati a large part of the problem is the LLM not being good enough to return the format correctly. We see this behavior more with running Llama3, Llama2, phi3 on Ollama. We see it less when you run these models on Groq. The issue is that Ollama LLM doesn't always return back the json function calling in the correct format i.e. it appends text before the function calls ("here is the json output: {}"). Let me do some digging and get back to you.

[francescoagati]

                                                                                                        
                                                                                                                       
DEBUG    Tool Calls: [                                                                                                 
           {                                                                                                           
             "type": "function",                                                                                       
             "function": {                                                                                             
               "name": "search_wikipedia",                                                                             
               "arguments": "{\"query\": \"Buddhism\"}"                                                                
             }                                                                                                         
           }                                                                                                           
         ]                                                                                                             
DEBUG    ============== user ==============                                                                            
DEBUG    {"content": "Buddhism ( BUUD-ih-z\u0259m, US also  BOOD-), also known as Buddha Dharma and Dharmavinaya, is an
         Indian religion and philosophical tradition based on teachings attributed to the Buddha. It is the world's    
         fourth-largest religion, with over 520 million followers, known as Buddhists, who comprise seven percent of   
         the global population. Buddhism originated in the eastern Gangetic plain as a \u015brama\u1e47a\u2013movement 
         in the 5th century BCE, and gradually spread throughout much of Asia via the Silk Road. \nAccording to        
         Buddhist tradition, the Buddha taught that attachment or clinging causes dukkha (often translated as          
         \"suffering\" or \"unease\"), but that there is a path of development which leads to awakening and full       
         liberation from dukkha. This path involves elements such as meditation and ethical precepts rooted in         
         non-harming, with the Buddha regarding his teachings as a Middle Way between extremes such as asceticism or   
         hedonism. Other widely observed elements include: the doctrines of dependent origination, the three marks of  
         existence, and karma; monasticism; the Three Jewels; and the cultivation of perfections                       
         (p\u0101ramit\u0101).\nBuddhist schools vary in their interpretation of the paths to liberation (m\u0101rga)  
         as well as the relative importance and \"canonicity\" assigned to various Buddhist texts, and their specific  
         teachings and practices. Two major extant branches of Buddhism are generally recognized by scholars:          
         Therav\u0101da (lit.\u2009'School of the Elders') and Mah\u0101y\u0101na (lit.\u2009'Great Vehicle'). The     
         Theravada tradition emphasizes the attainment of nirv\u0101\u1e47a (lit.\u2009'extinguishing') as a means of  
         transcending the individual self and ending the cycle of death and rebirth (sa\u1e43s\u0101ra), while the     
         Mahayana tradition emphasizes the Bodhisattva ideal, in which one works for the liberation of all sentient    
         beings. Additionally, Vajray\u0101na (lit.\u2009'Indestructible Vehicle'), a body of teachings incorporating  
         esoteric tantric techniques, may be viewed as a separate branch or tradition within Mah\u0101y\u0101na.\nThe  
         Buddhist canon is vast, with many different textual collections in different languages (such as Sanskrit,     
         Pali, Tibetan, and Chinese).  The Therav\u0101da branch has a widespread following in Sri Lanka as well as in 
         Southeast Asia, namely Myanmar, Thailand, Laos, and Cambodia. The Mah\u0101y\u0101na branch\u2014which        
         includes the traditions of Zen, Pure Land, Nichiren, Tiantai, Tendai, and Shingon\u2014is predominantly       
         practised in Nepal, Bhutan, China, Malaysia, Vietnam, Taiwan, Korea, and Japan. Tibetan Buddhism, which       
         preserves the Vajray\u0101na teachings of eighth-century India, is practised in the Himalayan states as well  
         as in Mongolia and Russian Kalmykia. Historically, until the early 2nd millennium, Buddhism was widely        
         practiced in the Indian subcontinent; it also had a foothold to some extent elsewhere in Asia, namely         
         Afghanistan, Turkmenistan, Uzbekistan, and Tajikistan.", "name": "Buddhism", "meta_data": {}}                 
DEBUG    ============== user ==============                                                                            
DEBUG    Using the results of the tools above, respond to the following message:                                       
                                                                                                                       
         <user_message>                                                                                                
         a long article with title description and body about buddhism?                                                
         </user_message>                                                                                               
DEBUG    ============== assistant ==============                                                                       
DEBUG    {                                                                                                             
             "tool_calls": [                                                                                           
                 {                                                                                                     
                     "name": "search_wikipedia",                                                                       
                     "arguments": {                                                                                    
                         "query": "Buddhism"                                                                           
                     }                                                                                                 
                 }                                                                                                     
             ]                                                                                                         
         }                                                                                                             
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
DEBUG    Tool Calls: [                                                                                                 
           {                                                                                                           
             "type": "function",                                                                                       
             "function": {                                                                                             
               "name": "search_wikipedia",                                                                             
               "arguments": "{\"query\": \"Buddhism\"}"                                                                
             }                                                                                                         
           }                                                                                                           
         ]                                                                                                             
DEBUG    ============== user ==============                                                                            
DEBUG    {"content": "Buddhism ( BUUD-ih-z\u0259m, US also  BOOD-), also known as Buddha Dharma and Dharmavinaya, is an
         Indian religion and philosophical tradition based on teachings attributed to the Buddha. It is the world's    
         fourth-largest religion, with over 520 million followers, known as Buddhists, who comprise seven percent of   
         the global population. Buddhism originated in the eastern Gangetic plain as a \u015brama\u1e47a\u2013movement 
         in the 5th century BCE, and gradually spread throughout much of Asia via the Silk Road. \nAccording to        
         Buddhist tradition, the Buddha taught that attachment or clinging causes dukkha (often translated as          
         \"suffering\" or \"unease\"), but that there is a path of development which leads to awakening and full       
         liberation from dukkha. This path involves elements such as meditation and ethical precepts rooted in         
         non-harming, with the Buddha regarding his teachings as a Middle Way between extremes such as asceticism or   
         hedonism. Other widely observed elements include: the doctrines of dependent origination, the three marks of  
         existence, and karma; monasticism; the Three Jewels; and the cultivation of perfections                       
         (p\u0101ramit\u0101).\nBuddhist schools vary in their interpretation of the paths to liberation (m\u0101rga)  
         as well as the relative importance and \"canonicity\" assigned to various Buddhist texts, and their specific  
         teachings and practices. Two major extant branches of Buddhism are generally recognized by scholars:          
         Therav\u0101da (lit.\u2009'School of the Elders') and Mah\u0101y\u0101na (lit.\u2009'Great Vehicle'). The     
         Theravada tradition emphasizes the attainment of nirv\u0101\u1e47a (lit.\u2009'extinguishing') as a means of  
         transcending the individual self and ending the cycle of death and rebirth (sa\u1e43s\u0101ra), while the     
         Mahayana tradition emphasizes the Bodhisattva ideal, in which one works for the liberation of all sentient    
         beings. Additionally, Vajray\u0101na (lit.\u2009'Indestructible Vehicle'), a body of teachings incorporating  
         esoteric tantric techniques, may be viewed as a separate branch or tradition within Mah\u0101y\u0101na.\nThe  
         Buddhist canon is vast, with many different textual collections in different languages (such as Sanskrit,     
         Pali, Tibetan, and Chinese).  The Therav\u0101da branch has a widespread following in Sri Lanka as well as in 
         Southeast Asia, namely Myanmar, Thailand, Laos, and Cambodia. The Mah\u0101y\u0101na branch\u2014which        
         includes the traditions of Zen, Pure Land, Nichiren, Tiantai, Tendai, and Shingon\u2014is predominantly       
         practised in Nepal, Bhutan, China, Malaysia, Vietnam, Taiwan, Korea, and Japan. Tibetan Buddhism, which       
         preserves the Vajray\u0101na teachings of eighth-century India, is practised in the Himalayan states as well  
         as in Mongolia and Russian Kalmykia. Historically, until the early 2nd millennium, Buddhism was widely        
         practiced in the Indian subcontinent; it also had a foothold to some extent elsewhere in Asia, namely         
         Afghanistan, Turkmenistan, Uzbekistan, and Tajikistan.", "name": "Buddhism", "meta_data": {}}                 
DEBUG    ============== user ==============                                                                            
DEBUG    Using the results of the tools above, respond to the following message:                                       
                                                                                                                       
         <user_message>                                                                                                
         a long article with title description and body about buddhism?                                                
         </user_message>                                                                                               
DEBUG    ============== assistant ==============                                                                       
DEBUG    {                                                                                                             
             "tool_calls": [                                                                                           
                 {                                                                                                     
                     "name": "search_wikipedia",                                                                       
                     "arguments": {                                                                                    
                         "query": "Buddhism"                                                                           
                     }                                                                                                 
                 }                                                                                                     
             ]                                                                                                         
         }                                                                                                             
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
DEBUG    Tool Calls: [                                                                                                 
           {                                                                                                           
             "type": "function",                                                                                       
             "function": {                                                                                             
               "name": "search_wikipedia",                                                                             
               "arguments": "{\"query\": \"Buddhism\"}"                                                                
             }                                                                                                         
           }                                                                                                           
         ]                                                                                                             
DEBUG    ============== user ==============                                                                            
DEBUG    {"content": "Buddhism ( BUUD-ih-z\u0259m, US also  BOOD-), also known as Buddha Dharma and Dharmavinaya, is an
         Indian religion and philosophical tradition based on teachings attributed to the Buddha. It is the world's    
         fourth-largest religion, with over 520 million followers, known as Buddhists, who comprise seven percent of   
         the global population. Buddhism originated in the eastern Gangetic plain as a \u015brama\u1e47a\u2013movement 
         in the 5th century BCE, and gradually spread throughout much of Asia via the Silk Road. \nAccording to        
         Buddhist tradition, the Buddha taught that attachment or clinging causes dukkha (often translated as          
         \"suffering\" or \"unease\"), but that there is a path of development which leads to awakening and full       
         liberation from dukkha. This path involves elements such as meditation and ethical precepts rooted in         
         non-harming, with the Buddha regarding his teachings as a Middle Way between extremes such as asceticism or   
         hedonism. Other widely observed elements include: the doctrines of dependent origination, the three marks of  
         existence, and karma; monasticism; the Three Jewels; and the cultivation of perfections                       
         (p\u0101ramit\u0101).\nBuddhist schools vary in their interpretation of the paths to liberation (m\u0101rga)  
         as well as the relative importance and \"canonicity\" assigned to various Buddhist texts, and their specific  
         teachings and practices. Two major extant branches of Buddhism are generally recognized by scholars:          
         Therav\u0101da (lit.\u2009'School of the Elders') and Mah\u0101y\u0101na (lit.\u2009'Great Vehicle'). The     
         Theravada tradition emphasizes the attainment of nirv\u0101\u1e47a (lit.\u2009'extinguishing') as a means of  
         transcending the individual self and ending the cycle of death and rebirth (sa\u1e43s\u0101ra), while the     
         Mahayana tradition emphasizes the Bodhisattva ideal, in which one works for the liberation of all sentient    
         beings. Additionally, Vajray\u0101na (lit.\u2009'Indestructible Vehicle'), a body of teachings incorporating  
         esoteric tantric techniques, may be viewed as a separate branch or tradition within Mah\u0101y\u0101na.\nThe  
         Buddhist canon is vast, with many different textual collections in different languages (such as Sanskrit,     
         Pali, Tibetan, and Chinese).  The Therav\u0101da branch has a widespread following in Sri Lanka as well as in 
         Southeast Asia, namely Myanmar, Thailand, Laos, and Cambodia. The Mah\u0101y\u0101na branch\u2014which        
         includes the traditions of Zen, Pure Land, Nichiren, Tiantai, Tendai, and Shingon\u2014is predominantly       
         practised in Nepal, Bhutan, China, Malaysia, Vietnam, Taiwan, Korea, and Japan. Tibetan Buddhism, which       
         preserves the Vajray\u0101na teachings of eighth-century India, is practised in the Himalayan states as well  
         as in Mongolia and Russian Kalmykia. Historically, until the early 2nd millennium, Buddhism was widely        
         practiced in the Indian subcontinent; it also had a foothold to some extent elsewhere in Asia, namely         
         Afghanistan, Turkmenistan, Uzbekistan, and Tajikistan.", "name": "Buddhism", "meta_data": {}}                 
DEBUG    ============== user ==============                                                                            
DEBUG    Using the results of the tools above, respond to the following message:                                       
                                                                                                                       
         <user_message>                                                                                                
         a long article with title description and body about buddhism?                                                
         </user_message>                                                                                               
DEBUG    ============== assistant ==============                                                                       
DEBUG    {                                                                                                             
             "tool_calls": [                                                                                           
                 {                                                                                                     
                     "name": "search_wikipedia",                                                                       
                     "arguments": {                                                                                    
                         "query": "Buddhism"                                                                           
                     }                                                                                                 
                 }                                                                                                     
             ]                                                                                                         
         }                                                                                                             
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
                                                                                                                       
DEBUG    Tool Calls: [                                                                                                 
           {                                                                                                           
             "type": "function",                                                                                       
             "function": {                                                                                             
               "name": "search_wikipedia",                                                                             
               "arguments": "{\"query\": \"Buddhism\"}"                                                                
             }                                                                                                         
           }                                                                                                           
         ]                                                                                                             
DEBUG    ============== user ==============                                                                            
DEBUG    {"content": "Buddhism ( BUUD-ih-z\u0259m, US also  BOOD-), also known as Buddha Dharma and Dharmavinaya, is an
         Indian religion and philosophical tradition based on teachings attributed to the Buddha. It is the world's    
         fourth-largest religion, with over 520 million followers, known as Buddhists, who comprise seven percent of   
         the global population. Buddhism originated in the eastern Gangetic plain as a \u015brama\u1e47a\u2013movement 
         in the 5th century BCE, and gradually spread throughout much of Asia via the Silk Road. \nAccording to        
         Buddhist tradition, the Buddha taught that attachment or clinging causes dukkha (often translated as          
         \"suffering\" or \"unease\"), but that there is a path of development which leads to awakening and full       
         liberation from dukkha. This path involves elements such as meditation and ethical precepts rooted in         
         non-harming, with the Buddha regarding his teachings as a Middle Way between extremes such as asceticism or   
         hedonism. Other widely observed elements include: the doctrines of dependent origination, the three marks of  
         existence, and karma; monasticism; the Three Jewels; and the cultivation of perfections                       
         (p\u0101ramit\u0101).\nBuddhist schools vary in their interpretation of the paths to liberation (m\u0101rga)  
         as well as the relative importance and \"canonicity\" assigned to various Buddhist texts, and their specific  
         teachings and practices. Two major extant branches of Buddhism are generally recognized by scholars:          
         Therav\u0101da (lit.\u2009'School of the Elders') and Mah\u0101y\u0101na (lit.\u2009'Great Vehicle'). The     
         Theravada tradition emphasizes the attainment of nirv\u0101\u1e47a (lit.\u2009'extinguishing') as a means of  
         transcending the individual self and ending the cycle of death and rebirth (sa\u1e43s\u0101ra), while the     
         Mahayana tradition emphasizes the Bodhisattva ideal, in which one works for the liberation of all sentient    
         beings. Additionally, Vajray\u0101na (lit.\u2009'Indestructible Vehicle'), a body of teachings incorporating  
         esoteric tantric techniques, may be viewed as a separate branch or tradition within Mah\u0101y\u0101na.\nThe  
         Buddhist canon is vast, with many different textual collections in different languages (such as Sanskrit,     
         Pali, Tibetan, and Chinese).  The Therav\u0101da branch has a widespread following in Sri Lanka as well as in 
         Southeast Asia, namely Myanmar, Thailand, Laos, and Cambodia. The Mah\u0101y\u0101na branch\u2014which        
         includes the traditions of Zen, Pure Land, Nichiren, Tiantai, Tendai, and Shingon\u2014is predominantly       
         practised in Nepal, Bhutan, China, Malaysia, Vietnam, Taiwan, Korea, and Japan. Tibetan Buddhism, which       
         preserves the Vajray\u0101na teachings of eighth-century India, is practised in the Himalayan states as well  
         as in Mongolia and Russian Kalmykia. Historically, until the early 2nd millennium, Buddhism was widely        
         practiced in the Indian subcontinent; it also had a foothold to some extent elsewhere in Asia, namely         
         Afghanistan, Turkmenistan, Uzbekistan, and Tajikistan.", "name": "Buddhism", "meta_data": {}}                 
DEBUG    ============== user ==============                                                                            
DEBUG    Using the results of the tools above, respond to the following message:                                       
                                                                                                                       
         <user_message>                                                                                                
         a long article with title description and body about buddhism?                                                
         </user_message>                                                                                               
DEBUG    Time to generate response: 2.0734s                                                                            
DEBUG    ============== assistant ==============                                                                       
DEBUG    {                                                                                                             
             "tool_calls": [                                                                                           
                 {                                                                                                     
                     "name": "search_wikipedia",                                                                       
                     "arguments": {                                                                                    
                         "query": "Buddhism"                                                                           
                     }                                                                                                 
                 }                                                                                                     
             ]                                                                                                         
         }                                                                                                             
                                                                                                                       
                                                                                                                       
         Follow-up Question 1:                                                                                         
         Can you provide a more in-depth analysis of the historical spread and influence of Buddhism, especially its   
         interaction with other major religions?                                                                       
                                                                                                                       
DEBUG    ---------- Ollama Response End ----------                                                                     
DEBUG    *********** Assistant Run End: 6309253f-e133-4202-8760-b6802090e44a ***********                               
╭──────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Message  │ a long article with title description and body about buddhism?                                           │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Response │                                                                                                          │
│ (18.4s)  │  - Running: search_wikipedia(query=Buddhism)                                                             │
│          │                                                                                                          │
│          │                                                                                                          │
│          │  - Running: search_wikipedia(query=Buddhism)                                                             │
│          │                                                                                                          │
│          │                                                                                                          │
│          │  - Running: search_wikipedia(query=Buddhism)                                                             │
│          │                                                                                                          │
│          │                                                                                                          │
│          │  - Running: search_wikipedia(query=Buddhism)                                                             │
│          │                                                                                                          │
│          │                                                                                                          │
│          │  - Running: search_wikipedia(query=Buddhism)                                                             │
│          │                                                                                                          │
│          │ {                                                                                                        │
│          │     "tool_calls": [                                                                                      │
│          │         {                                                                                                │
│          │             "name": "search_wikipedia",                                                                  │
│          │             "arguments": {                                                                               │
│          │                 "query": "Buddhism"                                                                      │
│          │             }                                                                                            │
│          │         }                                                                                                │
│          │     ]                                                                                                    │
│          │ }                                                                                                        │
│          │                                                                                                          │
│          │                                                                                                          │
│          │ Follow-up Question 1:                                                                                    │
│          │ Can you provide a more in-depth analysis of the historical spread and influence of Buddhism, especially  │
│          │ its interaction with other major religions?                                                              │
│          │           

[ysolanky]

Hey @francescoagati phi3 is a 3.8B parameter model. It is great for conversation but not so much for function calling or even RAG. You mentioned that phi3 has good support for function calling. Can you please share some examples of that so that we can look into it? Thanks

[francescoagati]

see this https://twitter.com/AndroidBlogger/status/1783068972486742229