How to use a local AI model to answer SMS messages

Downloading a GGUF model from Huggingface (video instructions)

The video below walks you through all the steps in motion, starting with navigating to the website, all the way to moving the LLM file to the right folder. It's a worthwhile use of your time, since it takes a mere 75 seconds to watch, but captures every important step in detail.

Downloading a GGUF model from Huggingface (step-by-step guide)

In your browser, enter huggingface.co in the searchbar. You should land on a page similar to Figure 1.

Open huggingface.co page
Figure 1 - Open huggingface.co page

First, click Models near the top edge of the screen. In the left panel, set the filter to Libraries, then select GGUF from the available options, as seen in Figure 2.

Select GGUF models
Figure 2 - Select GGUF models

Select your preferred LLM model, as shown in Figure 3. For the sake of this tutorial, we'll be using the following model:

Meta-Llama-3-3.1-8B-Instruct-hf-Q4_K_M-GGUF

Select model library
Figure 3 - Select model library

On the page of the model, open the Files and versions tab, and look for the file with a .gguf format, highlighted in red in Figure 4.

Download model file
Figure 4 - Download model file

Download the .gguf file, and move or paste it to the following folder (Figure 5):

C:\AIModels

Figure 5 - Copy GGUF model file to C:\AIModels

How to configure local AI chatbot in Ozeki SMS Gateway

Open AI studio
Figure 6 - Open AI studio

Create newm GGUF AI model
Figure 7 - Create newm GGUF AI model

Select model file
Figure 8 - Select model file

Open SMS Gateway
Figure 9 - Open SMS Gateway

Add newm user/application
Figure 10 - Add newm user/application

Install AI chatbot
Figure 11 - Install AI chatbot

Select AI model
Figure 12 - Select AI model

Enable chatbot
Figure 13 - Enable chatbot

How to configure SMPP client connection for AI SMS

Add new connection
Figure 14 - Add new connection

Install SMPP Client
Figure 15 - Install SMPP Client

Configure SMPP client
Figure 16 - Configure SMPP client

Enable logging
Figure 17 - Enable logging

SMPP Client connected
Figure 18 - SMPP Client connected

Create new incoming route
Figure 19 - Create new incoming route

Create new outgoing route
Figure 20 - Create new outgoing route

Send test SMS message what is answered by AI chatbot

Open AI chatbot
Figure 16 - Open AI chatbot

Configure system prompt
Figure 17 - Configure system prompt

Send SMS message
Figure 18 - Send SMS message

AI generated answer recived
Figure 19 - AI generated answer recived

More information