Artificial Intelligence (AI) Simplified - Part 1: Build a Local AI Chatbot with Java

 

“My laptop cannot connect to the office Wi-Fi.”

An IT support conversation often begins with a statement like this. Before troubleshooting, somebody must understand the problem and ask the right questions.

Could we build an assistant that starts that conversation on our own computer?

Let’s create Pixel, a conversational IT help-desk assistant using Java, Ollama, and a small Qwen model.

This is the first article in a three-part series:

1. Conversation: Build a local chatbot with a Java command-line interface.

2. Action: Connect the chatbot to an action-focused service.

3. Integration: Introduce MCP to expose reusable tools.

What We Are Building

Our first version has three components:

Component	Responsibility
Java CLI	Accepts input, retains recent conversation history, and displays responses
Ollama	Runs the model and exposes an HTTP API
Qwen3 0.6B	Generates responses from the supplied conversation

• Ollama runs in Docker. 
• The Java application runs directly on your computer and communicates with Ollama over HTTP.
• Pixel can discuss common IT problems, but this version cannot inspect your computer, check live service status, or create tickets. We will introduce those capabilities progressively.

Step 1: Prepare Your Environment

Install:

• Docker Desktop, or Docker Engine on Linux.

• JDK 17, including the Java compiler.

• Gradle 8.14.3, available from the Gradle releases page.

Use these practical resource allowances:

Resource	Starting point
Disk space	At least 5 GB free for the model, container image, Gradle, dependencies, and build files
Memory	8 GB RAM or more, with several gigabytes available to Docker
Compute	A modern CPU; a compatible GPU is optional
Internet	Required for initial downloads

These are planning allowances, rather than guaranteed minimum requirements. Existing Docker images, caches, and other applications may require more space and memory.

The model download is approximately 523 MB. Its runtime memory requirement is higher, and the complete development setup occupies additional disk space.

Open PowerShell on Windows, or Terminal on macOS/Linux, and run:

java -version

javac -version

gradle --version

docker version

A Note About Response Speed

This tutorial uses CPU-based inference. It avoids the additional drivers and configuration required for GPU acceleration.

A supported GPU, correctly configured for container access, can generate responses faster. Merely having a GPU in your computer does not mean this container will use it.

On a CPU-only setup, expect noticeable pauses. Response time also depends on available memory, processor capability, conversation length, and competing workloads. The first request can take longer because the model must load into memory.

Start with short conversations and close memory-intensive applications. Evaluate performance on the intended deployment hardware before drawing conclusions about the application’s potential.

Step 2: Start Ollama

Run this command in PowerShell or Bash:

docker run -d --name pixel-ollama -p 127.0.0.1:11434:11434 -v pixel-models:/root/.ollama ollama/ollama:latest

Here is what it does:

Argument	Purpose
-d	Runs the container in the background
--name pixel-ollama	Assigns a recognizable container name
-p 127.0.0.1:11434:11434	Makes the API accessible through your computer’s loopback address
-v pixel-models:/root/.ollama	Preserves downloaded models in a Docker volume

Check that the container is running:

docker ps --filter name=pixel-ollama

If you previously created this container, start it instead of creating another:

docker start pixel-ollama

The example uses the current latest image for convenience. For repeatable team deployments, record and pin a tested image version or digest.

Step 3: Download and Check the Model

Download Qwen3 0.6B:

docker exec -it pixel-ollama ollama pull qwen3:0.6b

Verify that it is available:

docker exec pixel-ollama ollama list

You should see qwen3:0.6b.

Now check that the model responds:

docker exec -it pixel-ollama ollama run qwen3:0.6b "Introduce yourself as Pixel, a friendly IT support assistant. Use two sentences."

The wording will vary. Allow extra time for the first response.

The model remains in the Docker volume, so you do not need to download it every time you restart the container.

Step 4: Create the Java Project

Create and enter the project directory:

mkdir pixel-cli

cd pixel-cli

Create the source directory.

Windows PowerShell:

New-Item -ItemType Directory -Force src/main/java

macOS/Linux:

mkdir -p src/main/java

Create settings.gradle in the pixel-cli directory:

rootProject.name = 'pixel-cli'

Create build.gradle alongside it:

plugins {
    id 'application'
}

repositories {
    mavenCentral()
}

dependencies {
    implementation 'com.fasterxml.jackson.core:jackson-databind:2.18.2'
}

java {
    sourceCompatibility = JavaVersion.VERSION_17
    targetCompatibility = JavaVersion.VERSION_17
}

application {
    mainClass = 'Pixel'
}

We use Java’s HTTP client to communicate with Ollama and Jackson to serialize and parse JSON.

Generate the Gradle Wrapper:

gradle wrapper --gradle-version 8.14.3

Subsequent build commands will use the project’s selected Gradle version.

Step 5: Write Pixel

Create src/main/java/Pixel.java:

import com.fasterxml.jackson.databind.ObjectMapper;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Scanner;

public class Pixel {

    private static final String SYSTEM_PROMPT = """
            You are Pixel, a friendly IT help-desk assistant.
            Help users describe and troubleshoot common IT problems.
            Ask one clarifying question at a time.
            Keep replies concise and easy to understand.
            Never ask for passwords, recovery codes, or other secrets.
            Do not claim to inspect systems, check live service status,
            create tickets, or perform actions.
            You have no tools or access to the user's computer.
            Admit uncertainty and recommend contacting IT when appropriate.
            """;

    private static final ObjectMapper JSON = new ObjectMapper();

    private static final HttpClient HTTP = HttpClient.newBuilder()
            .connectTimeout(Duration.ofSeconds(10))
            .build();

    public static void main(String[] args) {
        String model = args.length > 0 ? args[0] : "qwen3:0.6b";

        String baseUrl = System.getenv().getOrDefault(
                "OLLAMA_URL",
                "http://127.0.0.1:11434"
        ).replaceAll("/+$", "");

        URI endpoint = URI.create(baseUrl + "/api/chat");

        List<Map<String, String>> history = new ArrayList<>();

        System.out.println("Pixel | model: " + model);
        System.out.println("Describe an IT issue.");
        System.out.println("Commands: /reset, /exit");

        try (Scanner input = new Scanner(System.in)) {
            while (true) {
                System.out.print("\nYou: ");

                if (!input.hasNextLine()) {
                    break;
                }

                String text = input.nextLine().trim();

                if (text.equalsIgnoreCase("/exit")) {
                    break;
                }

                if (text.equalsIgnoreCase("/reset")) {
                    history.clear();
                    System.out.println("Pixel: Conversation cleared.");
                    continue;
                }

                if (text.isBlank()) {
                    continue;
                }

                if (text.length() > 1500) {
                    System.out.println(
                            "Pixel: Please keep each message "
                                    + "to 1,500 characters or fewer."
                    );
                    continue;
                }

                List<Map<String, String>> messages = new ArrayList<>();
                messages.add(message("system", SYSTEM_PROMPT));
                messages.addAll(history);
                messages.add(message("user", text));

                try {
                    String body = JSON.writeValueAsString(Map.of(
                            "model", model,
                            "messages", messages,
                            "stream", false,
                            "think", false,
                            "options", Map.of(
                                    "temperature", 0.3,
                                    "num_ctx", 4096,
                                    "num_predict", 200
                            )
                    ));

                    HttpRequest request = HttpRequest.newBuilder(endpoint)
                            .timeout(Duration.ofMinutes(3))
                            .header("Content-Type", "application/json")
                            .POST(HttpRequest.BodyPublishers.ofString(body))
                            .build();

                    System.out.println("Pixel is preparing a reply...");

                    HttpResponse<String> response = HTTP.send(
                            request,
                            HttpResponse.BodyHandlers.ofString()
                    );

                    if (response.statusCode() != 200) {
                        System.out.println(
                                "Ollama returned HTTP "
                                        + response.statusCode()
                                        + ": " + response.body()
                        );
                        continue;
                    }

                    String answer = JSON.readTree(response.body())
                            .path("message")
                            .path("content")
                            .asText("")
                            .trim();

                    if (answer.isEmpty()) {
                        System.out.println(
                                "Pixel: No reply received. Try again."
                        );
                        continue;
                    }

                    System.out.println("\nPixel: " + answer);

                    history.add(message("user", text));
                    history.add(message("assistant", answer));

                    // Retain at most four completed exchanges.
                    while (history.size() > 8) {
                        history.subList(0, 2).clear();
                    }

                } catch (InterruptedException exception) {
                    Thread.currentThread().interrupt();
                    System.out.println("Conversation interrupted.");
                    break;
                } catch (Exception exception) {
                    System.out.println(
                            "Could not complete the request: "
                                    + exception.getMessage()
                    );
                    System.out.println(
                            "Check that Ollama is running "
                                    + "and the model is downloaded."
                    );
                }
            }
        }

        System.out.println("Goodbye!");
    }

    private static Map<String, String> message(
            String role,
            String content
    ) {
        return Map.of("role", role, "content", content);
    }
}

Step 6: Build and Start the Chatbot

From the pixel-cli directory, build an application distribution.

Windows PowerShell:

.\gradlew.bat installDist

Launch Pixel:

.\build\install\pixel-cli\bin\pixel-cli.bat

macOS/Linux:

./gradlew installDist

Launch Pixel:

./build/install/pixel-cli/bin/pixel-cli

The first build downloads the required Gradle distribution and Java dependencies.

We launch the generated application directly so it receives terminal input without relying on Gradle to forward it.

You should see:

Pixel | model: qwen3:0.6b
Describe an IT issue.
Commands: /reset, /exit

You:

Step 7: Have a Conversation

Enter:

My Windows laptop cannot connect to the office Wi-Fi.

After Pixel responds, provide more context:

Other people can connect, but my laptop cannot see the network.

The application sends the recent conversation with each request, allowing Pixel to respond in context.

Try asking:

Explain your suggested check in simpler terms.

To start a fresh conversation:

/reset

To exit:

/exit

The history is held in memory and disappears when the application exits.

Step 8: Understand the Important Settings

The HTTP request contains these settings:

Setting	Purpose
model	Selects qwen3:0.6b by default
messages	Supplies the system prompt, recent conversation, and new message
stream: false	Returns the generated reply as one response
think: false	Disables Qwen’s separate thinking mode
temperature: 0.3	Encourages less varied responses
num_ctx: 4096	Sets the context window
num_predict: 200	Limits generated output

Lower temperature does not guarantee correctness. Similarly, limiting the number of exchanges bounds retained history, but does not guarantee that every conversation fits within the context window.

We have kept the example deliberately modest. Longer histories and larger context windows require more resources.

Step 9: Adjust Pixel’s Personality

Edit SYSTEM_PROMPT to add instructions such as:

Be warm and lightly witty, but keep troubleshooting clear.
Explain technical terms when speaking to a beginner.

Rebuild the application:

Windows:

.\gradlew.bat installDist

macOS/Linux:

./gradlew installDist

Then launch Pixel again.

You can also select a different downloaded model through a command-line argument:

Windows:

.\build\install\pixel-cli\bin\pixel-cli.bat qwen3:0.6b

macOS/Linux:

./build/install/pixel-cli/bin/pixel-cli qwen3:0.6b

To switch models, download the intended model first and replace the argument. Other models may require different request settings or substantially more memory.

Step 10: Stop and Resume

After exiting Pixel, stop Ollama:

docker stop pixel-ollama

Your downloaded model remains stored.

Next time, start Ollama:

docker start pixel-ollama

Then launch the generated Java application again. Rebuild only when you change the code or build configuration.

What Have We Established?

We now have a Java application that holds a conversation with a locally running model, maintains recent context, and accepts instructions defining its role.

For developers, this is a familiar HTTP integration. For students, it makes the relationship between an application and a language model concrete. For technology leaders, it provides a small, inspectable experiment before considering a broader investment.

Pixel’s responses remain generated suggestions. A small model can misunderstand a problem or give incorrect guidance, so use fictional support scenarios while you evaluate it.

The next milestone is more interesting: asking Pixel whether a service is available and having it obtain the answer from a real API.

In Part 2, we will connect the chatbot to a microservice and turn a conversation into a controlled action.

References

• Ollama Docker setup

• Ollama Chat API

• Qwen3 0.6B model

• Gradle releases