Integrating Ollama with DeepSeek-R1 in Spring Boot

Are you looking to leverage the power of Ollama and DeepSeek-R1 in your Spring Boot application? This post will walk you through the entire process, from understanding what Ollama is to implementing a seamless integration. 

What is Ollama?

Ollama is a powerful tool designed to simplify the deployment and management of large language models (LLMs) locally. It provides an easy-to-use API for interacting with models like DeepSeek-R1, making it an excellent choice for developers who want to integrate AI capabilities into their applications without relying on external cloud services.

With Ollama, you can:

• Run LLMs locally on your machine.
• Switch between different model versions effortlessly.
• Integrate AI capabilities into your applications via a simple API.

Why Integrate Ollama with DeepSeek-R1?

DeepSeek-R1 is a state-of-the-art language model that offers high performance and flexibility. By integrating it with Ollama in your Spring Boot application, you can:

• Build AI-powered features like chatbots, content generators, and more.
• Keep your AI logic local, ensuring data privacy and reducing latency.
• Easily switch between different versions of DeepSeek-R1 based on your application’s needs.

Step 1: Install Ollama

To get started, you’ll need to install Ollama on your system. Run the following command in your terminal:

curl -fsSL https://ollama.com/install.sh | sh

Successful Installation Output:

>>> Cleaning up old version
>>> Installing ollama to /usr/local
>>> Downloading Linux amd64 bundle
>>> Creating ollama user...
>>> Adding ollama user to groups...
>>> Creating ollama systemd service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service
>>> Nvidia GPU detected
>>> API available at 127.0.0.1:11434

Once installed, Ollama will be ready to use, and the API will be available at http://localhost:11434.

Step 2: Application Configuration

Next, configure your Spring Boot application by updating the application.yml file:

spring:
  application:
    name: demo-deepseek-r1.ollama

# Server configuration
server:
  port: 8080
  error:
    include-message: always

# Ollama configuration
ollama:
  endpoint: http://localhost:11434/api/generate
  model: deepseek-r1:1.5b
  timeout:
    connect: 30000
    read: 60000

This configuration sets up the Ollama endpoint, model, and timeout settings for your application.

Step 3: Core Implementation

Create the following records to handle requests and responses:

// OllamaRequest.java
@JsonInclude(JsonInclude.Include.NON_NULL)
public record OllamaRequest(
    String model,
    String prompt,
    boolean stream
) {}

// OllamaResponse.java
@JsonIgnoreProperties(ignoreUnknown = true)
public record OllamaResponse(
    String model,
    String response,
    String created_at,
    boolean done
) {}

Service Layer

Implement the OllamaService to interact with the Ollama API:

@Service
public class OllamaService {

    private static final String OLLAMA_API_URL = "http://localhost:11434/api/generate";
    private final RestTemplate restTemplate;

    public OllamaService(RestTemplateBuilder restTemplateBuilder) {
        this.restTemplate = restTemplateBuilder
                .build();
    }

    public String generateResponse(String prompt) {
        try {
            OllamaRequest request = new OllamaRequest("deepseek-r1:1.5b", prompt, false);
            HttpHeaders headers = new HttpHeaders();
            headers.setContentType(MediaType.APPLICATION_JSON);

            ResponseEntity<OllamaResponse> response = restTemplate.exchange(
                    OLLAMA_API_URL,
                    HttpMethod.POST,
                    new HttpEntity<>(request, headers),
                    OllamaResponse.class
            );

            if (response.getStatusCode().is2xxSuccessful() && response.getBody() != null) {
                return response.getBody().response() != null
                        ? response.getBody().response()
                        : "Received empty response from model";
            }
            return "Ollama API returned status: " + response.getStatusCode();
        } catch (RestClientException e) {
            return "Error communicating with Ollama: " + e.getMessage();
        }
    }
}

REST Controller

Create a REST controller to expose the chat endpoint:

@RestController
@RequestMapping("/api/chat")
public class ChatController {

    private final OllamaService ollamaService;

    public ChatController(OllamaService ollamaService) {
        this.ollamaService = ollamaService;
    }

    @PostMapping
    public ResponseEntity<String> chat(@RequestBody String prompt) {
        if (prompt == null || prompt.isBlank()) {
            return ResponseEntity.badRequest().body("Prompt cannot be empty");
        }
        String response = ollamaService.generateResponse(prompt);
        return ResponseEntity.ok(response);
    }
}

Model Version Compatibility

Here’s a quick reference for DeepSeek-R1 model versions and their requirements:

*Check official model availability at:

Ollama Model Library

Testing the Integration

To test the integration, use the following curl  command or postman:

curl -X POST -H "Content-Type: text/plain" -d "Explain AI in simple terms" http://localhost:8080/api/chat

Ouput

Source Code

Here on GitHub.