Bringing AI Offline: Llama Local Models for Chrome Extensions

What is LLama ?

In simple terms, LLaMA (Large Language Model Meta AI) is a powerful computer program developed by Meta (the company formerly known as Facebook) that can understand and generate human language. It’s like a very advanced chatbot or text assistant.

What is Ollama ?

Ollama is a user-friendly tool that helps you run and manage AI models, which are computer programs designed to understand and generate human language. Think of Ollama as a smart assistant that makes it easier to use these advanced AI models without needing deep technical knowledge.

How LLama works ?

LLaMA is trained on a vast amount of text data from books, articles, websites, and other sources. It uses this training to learn patterns in language, such as grammar, vocabulary, and context. When you give LLaMA a task (like writing a story or answering a question), it uses what it has learned to produce a response that makes sense based on the input it receives.

In essence, LLaMA is like a super-smart text assistant that can handle a wide range of language-related tasks, making it a valuable tool for businesses, educators, writers, and anyone who works with language.

OpenTalkGpt Exntention Flow

Step 1 : Download ollama uing below link based on os :

•  To download in window click here 

•  To download in mac click here 

•  For linux run below command : 

curl -fsSL https://ollama.com/install.sh | sh

Once download done verify ollama is running or not with below link :

•  http://localhost:11434/api/ps //Show modal which is load in memory

  

  Modal Loaded In Memory

  
  

• http://localhost:11434/api/tags //Show downloaded modal

  

  Downloaded Modal

  
  

Step 2 : Download the OpenTalkGpt chrome extention

OpenTalkGPT Chrome Extension Effortlessly manage and interact with powerful open-source models like LLaMA right in your browser! OpenTalkGPT hosts an Ollama UI on your localhost, making it simple to explore and control models without complex setups. Perfect for seamless, intuitive AI experiences.

Download the OpenTalkGpt extention from here.

After download you can see the below screen and you can download and select modal from settings.

OpenTalkGpt Chrome Extention

Step 3 : Download the llama3/phi modal using extention.

Now go to Settings > Manage Modal and download the modal based on your requirement and use. It is recomended to download the llama 3.2 1b modal.

Manage Modal

Step 4 : Select the modal and start using it

Now go to Setting >Manage Settings and slect the moal which you want to use.

Modal Settings

Dive into the world of AI with Llama and Phi using OpenTalkGPT – all offline! 💻 Enjoy seamless interactions with cutting-edge models while keeping your data 🔒 100% secure on your device. 🌟 Explore the future of AI, your way! ✨

Daily used command of ollama for linux : 

• To run a specific model, you can use the following command:

 ollama run llama3.2

• To see the list of available models, use:

ollama list

• To see ollama version : 

ollama --version

• To download modal: 

ollama pull llama3

• List which models are currently loaded:

ollama ps

• Stop a model which is currently running

ollama stop llama3.2

• To check ollama service status

systemctl status ollama