How to use a local AI model to answer Whatsapp messages

Downloading a GGUF model from Huggingface (video instructions)

The video below walks you through all the steps in motion, starting with navigating to the website, all the way to moving the LLM file to the right folder. It's a worthwhile use of your time, since it takes a mere 75 seconds to watch, but captures every important step in detail.

Downloading a GGUF model from Huggingface (step-by-step guide)

In your browser, enter huggingface.co in the searchbar. You should land on a page similar to Figure 1.

Open huggingface.co page
Figure 1 - Open huggingface.co page

First, click Models near the top edge of the screen. In the left panel, set the filter to Libraries, then select GGUF from the available options, as seen in Figure 2.

Select GGUF models
Figure 2 - Select GGUF models

Select your preferred LLM model, as shown in Figure 3. For the sake of this tutorial, we'll be using the following model:

Meta-Llama-3-3.1-8B-Instruct-hf-Q4_K_M-GGUF

Select model library
Figure 3 - Select model library

On the page of the model, open the Files and versions tab, and look for the file with a .gguf format, highlighted in red in Figure 4.

Download model file
Figure 4 - Download model file

Download the .gguf file, and move or paste it to the following folder (Figure 5):

C:\AIModels

Figure 5 - Copy GGUF model file to C:\AIModels

How to configure local AI chatbot in Ozeki SMS Gateway

Open AI studio
Figure 6 - Open AI studio

Create newm GGUF AI model
Figure 7 - Create newm GGUF AI model

Select model file
Figure 8 - Select model file

Open SMS Gateway
Figure 9 - Open SMS Gateway

Add newm user/application
Figure 10 - Add newm user/application

Install AI chatbot
Figure 11 - Install AI chatbot

Select AI model
Figure 12 - Select AI model

Enable chatbot
Figure 13 - Enable chatbot

How to configure Whatsapp account connection for AI messaging

Detailed Whatsapp configuration:
How to setup a WhatsApp message connection
Receive WhatsApp messages in SMS Gateway

Open SMS Gateway
Figure 14 - Open SMS Gateway

Add new connection
Figure 15 - Add new connection

Install Whatsapp account
Figure 16 - Install Whatsapp account

Configure Whatsapp credentials
Figure 17 - Configure Whatsapp credentials

Configure defalt template
Figure 18 - Configure defalt template

Set telephone number
Figure 19 - Set telephone number

Whatsapp account connected
Figure 20 - Whatsapp account connected

Create new incoming route
Figure 21 - Create new incoming route

Create new outgoing route
Figure 22 - Create new outgoing route

Send Whatsapp test message what is answered by AI chatbot

Open AI chat bot
Figure 23 - Open AI chat bot

Configure system prompt
Figure 24 - Configure system prompt

Send Whatsapp message
Figure 25 - Send Whatsapp message

Generated response received
Figure 26 - Generated response received

More information