Testing a Spring Boot 3.2.5 App from Top to Bottom: MySQL, Docker, Unit Tests, Integration Tests

In this comprehensive guide, we'll walk you through setting up a Spring Boot 3.2.5 application with a MySQL database using Docker Compose and writing unit tests for various layers of the application. We'll cover the following topics:

1. Setting up the MySQL container using Docker Compose
2. Configuring the Spring Boot application
3. Defining the model, repository, service, and controller
4. Writing unit tests for the repository layer
5. Writing unit tests for the service layer
6. Writing unit tests for the controller layer
7. Writing integration tests

1. Setting up the MySQL container using Docker Compose

To start with, we'll set up a MySQL container using Docker Compose. This will allow us to run MySQL in a containerized environment, making it easy to manage and integrate with our Spring Boot application.

Create a docker-compose-mysql.yml file in your project root:

version: '2'
services:

  ### Mysql container
  mysql:
    image: mysql:latest
    ports:
      - "3306:3306"
    volumes:
      - /var/lib/mysql:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: mypass
      MYSQL_DATABASE: test_db
      MYSQL_USER: test
      MYSQL_PASSWORD: test_pass
      MYSQL_ROOT_HOST: '%'  # needs to be enclosed with quotes

Run the following command to start the MySQL container:

docker-compose -f docker-compose-mysql.yml up

2. Configuring the Spring Boot application

Next, configure the Spring Boot application to connect to the MySQL database. Add the following dependencies to your pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>3.2.5</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.henry</groupId>
	<artifactId>demo-testing</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>demo-testing</name>
	<description>Demo Spring Boot Unit Testing and Integration Testing</description>
	<properties>
		<java.version>21</java.version>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<!-- Mysql Connector -->
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<version>8.0.30</version>
			<scope>runtime</scope>
		</dependency>
		<!--<dependency>
			<groupId>com.h2database</groupId>
			<artifactId>h2</artifactId>
			<scope>runtime</scope>
		</dependency>-->
		<dependency>
			<groupId>org.projectlombok</groupId>
			<artifactId>lombok</artifactId>
			<optional>true</optional>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
				<configuration>
					<excludes>
						<exclude>
							<groupId>org.projectlombok</groupId>
							<artifactId>lombok</artifactId>
						</exclude>
					</excludes>
				</configuration>
			</plugin>
		</plugins>
	</build>

</project>

Update the application.yaml file to configure the MySQL connection:

spring:
  application:
   name: demo-testing
  jpa:
    show-sql: true
    hibernate:
      ddl-auto: update
  datasource:
      url: jdbc:mysql://localhost:3306/test_db?allowPublicKeyRetrieval=true
      username: test
      password: test_pass
      driver-class-name: com.mysql.cj.jdbc.Driver

3. Defining the model, repository, service, and controller

Define an Employee model class:

package com.henry.demotesting.model;

import jakarta.persistence.*;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
@Entity
@Table(name = "employees")
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "first_name", nullable = false)
    private String firstName;

    @Column(name = "last_name", nullable = false)
    private String lastName;

    @Column(nullable = false)
    private String email;
}

Define the EmployeeRepository interface:

package com.henry.demotesting.repository;

import com.henry.demotesting.model.Employee;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;

import java.util.Optional;

public interface EmployeeRepository extends JpaRepository<Employee, Long> {

    Optional<Employee> findByEmail(String email);

    @Query("select e from Employee e where e.firstName = ?1 and e.lastName = ?2")
    Employee findByJPQL(String firstName, String lastName);

    @Query("select e from Employee e where e.firstName =:firstName and e.lastName =:lastName")
    Employee findByJPQLNameParams(@Param("firstName") String firstName, @Param("lastName") String lastName);

    @Query(value = "select * from employees e where e.first_name = ?1 and e.last_name = ?2", nativeQuery = true)
    Employee findByNativeSQL(String firstName, String lastName);

    @Query(value = "select * from employees e where e.first_name =:firstName and e.last_name =:lastName", nativeQuery = true)
    Employee findByNativeSQLWithNameParams(@Param("firstName") String firstName, @Param("lastName") String lastName);
}

Create the EmployeeService interface and its implementation:

package com.henry.demotesting.service;

import com.henry.demotesting.model.Employee;

import java.util.List;
import java.util.Optional;

public interface EmployeeService {
    Employee saveEmployee(Employee employee);
    List<Employee> getEmployees();
    Optional<Employee> findById(Long id);
    Employee updateEmployee(Employee employee);
    void deleteEmployee(long id);

}

package com.henry.demotesting.service.impl;

import com.henry.demotesting.exception.ResourceNotFoundException;
import com.henry.demotesting.model.Employee;
import com.henry.demotesting.repository.EmployeeRepository;
import com.henry.demotesting.service.EmployeeService;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.Optional;

@Service
public class EmployeeServiceImpl implements EmployeeService {

    private  final EmployeeRepository employeeRepository;

    public EmployeeServiceImpl(EmployeeRepository employeeRepository) {
        this.employeeRepository = employeeRepository;
    }

    @Override
    public Employee saveEmployee(Employee employee)  {

        Optional<Employee> savedEmployee = employeeRepository.findByEmail(employee.getEmail());
        if(savedEmployee.isPresent()){
            throw new ResourceNotFoundException("Employee already exist with given email: "+employee.getEmail());
        }
        return employeeRepository.save(employee);
    }

    @Override
    public List<Employee> getEmployees() {
        return employeeRepository.findAll();
    }

    @Override
    public Optional<Employee> findById(Long id) {
        return employeeRepository.findById(id);
    }

    @Override
    public Employee updateEmployee(Employee employee) {
        return employeeRepository.save(employee);
    }

    @Override
    public void deleteEmployee(long id) {
        employeeRepository.deleteById(id);
    }
}

Create the EmployeeController class:

package com.henry.demotesting.controller;

import com.henry.demotesting.model.Employee;
import com.henry.demotesting.service.EmployeeService;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.util.List;

@RestController
@RequestMapping("/api/employees")
public class EmployeeController {

    private final EmployeeService employeeService;

    public EmployeeController(EmployeeService employeeService) {
        this.employeeService = employeeService;
    }

    @PostMapping
    @ResponseStatus(HttpStatus.CREATED)
    public Employee createEmployee(@RequestBody Employee employee){
        return employeeService.saveEmployee(employee);
    }

    @GetMapping
    public List<Employee> getAllEmployees(){
        return employeeService.getEmployees();
    }

    @GetMapping("/{id}")
    public ResponseEntity<Employee> findEmployeeId(@PathVariable Long id){
        return employeeService.findById(id)
                .map(ResponseEntity::ok)
                .orElseGet(()-> ResponseEntity.notFound().build());
    }

    @PutMapping("/{id}")
    public ResponseEntity<Employee> updateEmployee(@PathVariable long id,
                                                   @RequestBody Employee employee){
        return employeeService.findById(id)
                .map(savedEmployee -> {
                    savedEmployee.setFirstName(employee.getFirstName());
                    savedEmployee.setLastName(employee.getLastName());
                    savedEmployee.setEmail(employee.getEmail());
                    var updEmployee = employeeService.updateEmployee(savedEmployee);
                    return  new ResponseEntity<>(updEmployee, HttpStatus.OK);
                })
                .orElseGet(() -> ResponseEntity.notFound().build());

    }

    @DeleteMapping("/{id}")
    public ResponseEntity<String> deleteEmployee(@PathVariable long id){
        employeeService.deleteEmployee(id);
        return new ResponseEntity<String>("Employee deleted successfully!", HttpStatus.OK);
    }
}

4. Writing unit tests for the repository layer

Create a test class for EmployeeRepository:

package com.henry.demotesting.repository;


import com.henry.demotesting.model.Employee;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;

import java.util.Optional;

import static org.assertj.core.api.AssertionsForInterfaceTypes.assertThat;


@DataJpaTest
public class EmployeeRepositoryTests {

    @Autowired
    private EmployeeRepository employeeRepository;

    //JUnit test for save employee operation
    @DisplayName("JUnit test for save employee operation")
    @Test
    public  void givenEmployeeObject_whenSave_thenReturnSavedEmployee(){

        //given  - precondition or setup
        var employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();

        //when - action or the behaviour that we are going test
        var savedEmployee = employeeRepository.save(employee);

        // then - verify the output
        assertThat(savedEmployee).isNotNull();
        assertThat(savedEmployee.getId()).isGreaterThan(0);

    }

     //JUnit test for get all employees operation
    @DisplayName("JUnit test for get all employees operation")
     @Test
     public  void givenEmployeeList_whenFindAll_thenEmployeeList(){

         //given  - precondition or setup
         var employee = Employee.builder()
                 .firstName("Henry")
                 .lastName("x")
                 .email("test@gmail.com")
                 .build();

         var employee1 = Employee.builder()
                 .firstName("Henry")
                 .lastName("x")
                 .email("test@gmail.com")
                 .build();

         employeeRepository.save(employee);
         employeeRepository.save(employee1);

         //when - action or the behaviour that we are going test
         var employeeList = employeeRepository.findAll();

         // then - verify the output
         assertThat(employeeList).isNotNull();
         assertThat(employeeList.size()).isEqualTo(2);

     }

     //JUnit test for get employee id operation
    @DisplayName("JUnit test for get employee id operation")
    @Test
    public  void givenEmployeeObject_whenFindById_thenReturnEmployeeObject(){

         //given  - precondition or setup
         var employee = Employee.builder()
                 .firstName("Henry")
                 .lastName("x")
                 .email("test@gmail.com")
                 .build();

         employeeRepository.save(employee);

         //when - action or the behaviour that we are going test
          var employeeDB = employeeRepository.findById(employee.getId()).get();

         // then - verify the output
         assertThat(employeeDB).isNotNull();
     }

     //JUnit test for get employee email operation
    @DisplayName("JUnit test for get employee email operation")
     @Test
     public  void givenEmployeeObject_whenFindByEmail_thenReturnEmployeeObject(){

         //given  - precondition or setup
        var employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
        employeeRepository.save(employee);

         //when - action or the behaviour that we are going test
        var employeeDB = employeeRepository.findByEmail(employee.getEmail()).get();

         // then - verify the output
        assertThat(employeeDB).isNotNull();
     }

    //JUnit test for update employee operation
    @DisplayName("JUnit test for update employee operation")
    @Test
    public  void givenEmployeeObject_whenUpdateEmployee_thenReturnUpdateEmployee(){

        //given  - precondition or setup
        var employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
        employeeRepository.save(employee);

        //when - action or the behaviour that we are going test
        var savedEmployee = employeeRepository.findById(employee.getId()).get();
        savedEmployee.setEmail("henry@gmail.com");
        savedEmployee.setFirstName("henry2");

        var updateEmployee = employeeRepository.save(savedEmployee);

        // then - verify the output
        assertThat(updateEmployee.getEmail()).isEqualTo("henry@gmail.com");
        assertThat(updateEmployee.getFirstName()).isEqualTo("henry2");
    }

    //JUnit test for delete employee operation
    @DisplayName("JUnit test for delete employee operation")
    @Test
    public  void givenEmployeeObject_whenDelete_thenRemoveEmployee(){

        //given  - precondition or setup
        var employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
        employeeRepository.save(employee);

        //when - action or the behaviour that we are going test
        employeeRepository.delete(employee);
        Optional<Employee> employeeOptional = employeeRepository.findById(employee.getId());

        // then - verify the output
        assertThat(employeeOptional).isEmpty();

    }

    //JUnit test for custom query using JPQL with index
    @DisplayName("JUnit test for custom query using JPQL with index")
    @Test
    public  void givenFirstNameAndLastName_whenFindByJPQL_thenEmployeeObject(){

        //given  - precondition or setup
        var employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
        employeeRepository.save(employee);
        var firstName = "Henry";
        var lastName = "x";

        //when - action or the behaviour that we are going test
        Employee savedEmployee = employeeRepository.findByJPQL(firstName, lastName);

        // then - verify the output
        assertThat(savedEmployee).isNotNull();
    }

    //JUnit test for custom query using JPQL with nameParams
    @DisplayName("JUnit test for custom query using JPQL with nameParams")
    @Test
    public  void givenFirstNameAndLastName_whenFindByJPQLNamedParams_thenEmployeeObject(){

        //given  - precondition or setup
        var employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
        employeeRepository.save(employee);
        var firstName = "Henry";
        var lastName = "x";

        //when - action or the behaviour that we are going test
        Employee savedEmployee = employeeRepository.findByJPQLNameParams(firstName, lastName);

        // then - verify the output
        assertThat(savedEmployee).isNotNull();

    }

    //JUnit test for using Nativa SQL with index params
    @DisplayName("JUnit test for using Nativa SQL with index params")
    @Test
    public  void givenFirstNameAndLastName_whenFindByNativeSQL_thenEmployeeObject(){

        //given  - precondition or setup
        var employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
        employeeRepository.save(employee);
        var firstName = "Henry";
        var lastName = "x";

        //when - action or the behaviour that we are going test
        Employee savedEmployee = employeeRepository.findByNativeSQL(firstName, lastName);

        // then - verify the output
        assertThat(savedEmployee).isNotNull();

    }

    //JUnit test for using Nativa SQL with named params
    @DisplayName("JUnit test for using Nativa SQL with named params")
    @Test
    public  void givenFirstNameAndLastName_whenFindByNativeSQLWithParams_thenEmployeeObject(){

        //given  - precondition or setup
        var employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
        employeeRepository.save(employee);
        var firstName = "Henry";
        var lastName = "x";

        //when - action or the behaviour that we are going test
        Employee savedEmployee = employeeRepository.findByNativeSQLWithNameParams(firstName, lastName);

        // then - verify the output
        assertThat(savedEmployee).isNotNull();

    }
}

5. Writing unit tests for the service layer

Create a test class for EmployeeService:

package com.henry.demotesting.service;


import com.henry.demotesting.exception.ResourceNotFoundException;
import com.henry.demotesting.model.Employee;
import com.henry.demotesting.repository.EmployeeRepository;
import com.henry.demotesting.service.impl.EmployeeServiceImpl;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;


import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static  org.mockito.BDDMockito.given;
import static org.mockito.BDDMockito.willDoNothing;
import static org.mockito.Mockito.*;

import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;

import java.util.Collections;
import java.util.List;
import java.util.Optional;

@ExtendWith(MockitoExtension.class)
public class EmployeeServiceTests {

    @Mock
    private EmployeeRepository employeeRepository;
    @InjectMocks
    private EmployeeServiceImpl employeeService;

    private Employee employee;

    @BeforeEach
    public  void setup(){
       /***
        * employeeRepository = Mockito.mock(EmployeeRepository.class);
        employeeService = new EmployeeServiceImpl(employeeRepository);
        **/
        employee = Employee.builder()
                .id(1L)
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
    }

    //JUnit test for savedEmployee method
    @DisplayName("JUnit test for savedEmployee method")
    @Test
    public  void givenEmployeeObject_whenSavedEmployee_thenReturnEmployeeObject(){

        //given  - precondition or setup
       given(employeeRepository.findByEmail(employee.getEmail()))
                .willReturn(Optional.empty());

      given(employeeRepository.save(employee)).willReturn(employee);

        //when - action or the behaviour that we are going test
        Employee savedEmployee = employeeService.saveEmployee(employee);

        // then - verify the output
        assertThat(savedEmployee).isNotNull();
    }

    //JUnit test for savedEmployee method which throws exception
    @DisplayName("JUnit test for savedEmployee method which throws exception")
    @Test
    public  void givenExistingEmail_whenSavedEmployee_thenThrowsException(){

        //given  - precondition or setup
        given(employeeRepository.findByEmail(employee.getEmail()))
                .willReturn(Optional.of(employee));

        //when - action or the behaviour that we are going test
        org.junit.jupiter.api.Assertions.assertThrows(ResourceNotFoundException.class, () -> {
            employeeService.saveEmployee(employee);
        });

        // then - verify the output
        verify(employeeRepository, never()).save(any(Employee.class));
    }

    //JUnit test for getAllEmployees method
    @DisplayName("JUnit test for getAllEmployees method")
    @Test
    public  void givenEmployeesList_whenGetAllEmployees_thenReturnEmployeesList(){

        //given  - precondition or setup

        var employee1 = Employee.builder()
                .id(2L)
                .firstName("Henry1")
                .lastName("x1")
                .email("test1@gmail.com")
                .build();

        given(employeeRepository.findAll()).willReturn(List.of(employee, employee1));

        //when - action or the behaviour that we are going test
       var employeeList = employeeService.getEmployees();

        // then - verify the output
        assertThat(employeeList).isNotNull();
        assertThat(employeeList.size()).isEqualTo(2);
    }

    //JUnit test for getAllEmployees method
    @DisplayName("JUnit test for getAllEmployees method (negative scenario)")
    @Test
    public  void givenEmptyEmployeesList_whenGetAllEmployees_thenReturnEmptyEmployeesList(){

        //given  - precondition or setup
        var employee1 = Employee.builder()
                .id(2L)
                .firstName("Henry1")
                .lastName("x1")
                .email("test1@gmail.com")
                .build();

        given(employeeRepository.findAll()).willReturn(Collections.emptyList());

        //when - action or the behaviour that we are going test
        var employeeList = employeeService.getEmployees();

        // then - verify the output
        assertThat(employeeList).isEmpty();
        assertThat(employeeList.size()).isEqualTo(0);
    }

    //JUnit test for getEmployeeById method
    @DisplayName("JUnit test for getEmployeeById method")
    @Test
    public  void givenEmployeeId_whenGetEmployeeId_thenEmployeeObject(){

        //given  - precondition or setup
        given(employeeRepository.findById(employee.getId())).willReturn(Optional.of(employee));
        //when - action or the behaviour that we are going test
        var employeeObject = employeeService.findById(employee.getId());
        // then - verify the output
        assertThat(employeeObject).isNotEmpty();
    }

    //JUnit test for updateEmployee method
    @DisplayName("JUnit test for updateEmployee method")
    @Test
    public  void givenEmployeeObject_whenUpdateEmployee_thenReturnUpdateEmployee(){

        //given  - precondition or setup
        given(employeeRepository.save(employee)).willReturn(employee);
        employee.setEmail("henry2@test.com");
        employee.setFirstName("test2");
        employee.setLastName("x3");
       //when - action or the behaviour that we are going test
        var updateEmployee = employeeService.updateEmployee(employee);

        // then - verify the output
        assertThat(updateEmployee.getEmail()).isEqualTo("henry2@test.com");
        assertThat(updateEmployee.getFirstName()).isEqualTo("test2");
        assertThat(updateEmployee.getLastName()).isEqualTo("x3");
    }

   //JUnit test for deleteEmployee method
    @DisplayName("JUnit test for deleteEmployee method")
    @Test
    public  void givenEmployeeObject_whenDeleteEmployee_thenReturnDeleteObject(){
        long employeeId = 1L;
        //given  - precondition or setup
        willDoNothing().given(employeeRepository).deleteById(employeeId);

        //when - action or the behaviour that we are going test
        employeeService.deleteEmployee(employeeId);

        // then - verify the output
        verify(employeeRepository, times(1)).deleteById(employeeId);
    }
}

6. Writing unit tests for the controller layer

Create a test class for EmployeeController:

package com.henry.demotesting.controller;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.henry.demotesting.model.Employee;
import com.henry.demotesting.service.EmployeeService;
import org.hamcrest.CoreMatchers;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentMatchers;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;

import java.util.ArrayList;
import java.util.List;
import java.util.Optional;

import static org.mockito.BDDMockito.given;
import static org.mockito.BDDMockito.willDoNothing;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@WebMvcTest
public class EmployeeControllerTests {

    @Autowired
    private MockMvc  mockMvc;

    @MockBean
    private EmployeeService employeeService;

    @Autowired
    private ObjectMapper objectMapper;

    private Employee employee;

    @BeforeEach
    public  void setup(){
        employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
    }

    //JUnit test for createEmployee
    @DisplayName("JUnit test for create Employee")
    @Test
    public  void givenEmployeeObject_whenCreateEmployee_thenReturnSavedEmployee() throws Exception {

        //given  - precondition or setup
        given(employeeService.saveEmployee(ArgumentMatchers.any(Employee.class)))
                .willAnswer(invocation -> invocation.getArgument(0));

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(post("/api/employees")
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(employee))
        );

        // then - verify the output
        response.andDo(print())
                .andExpect(status().isCreated())
                .andExpect(jsonPath("$.firstName", CoreMatchers.is(employee.getFirstName())))
                .andExpect(jsonPath("$.lastName", CoreMatchers.is(employee.getLastName())))
                .andExpect(jsonPath("$.email", CoreMatchers.is(employee.getEmail())));
    }

    //JUnit test for getAllEmployees method
    @DisplayName("Junit test for getAllEmployees method")
    @Test
    public  void givenListOfEmployees_whenGetAllEmployees_thenReturnEmployeesList() throws Exception {

        var employee1 = Employee.builder()
                .firstName("Henry1")
                .lastName("x1")
                .email("test1@gmail.com")
                .build();

        //given  - precondition or setup
        List<Employee> listOfEmployees = new ArrayList<>();
        listOfEmployees.add(employee);
        listOfEmployees.add(employee1);

        given(employeeService.getEmployees()).willReturn(listOfEmployees);

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(get("/api/employees"));

        // then - verify the output
        response.andExpect(status().isOk())
                .andDo(print())
                .andExpect(jsonPath("$.size()"
                        , CoreMatchers.is(listOfEmployees.size())));

    }

    //JUnit test for getEmployeeById method
    @DisplayName("JUnit test for getEmployeeById method (positive scenario)")
    @Test
    public  void givenEmployeeId_whenGetEmployeeId_thenReturnEmployeeObject() throws Exception {

        long employeeId = 1L;
        //given  - precondition or setup
        given(employeeService.findById(employeeId)).willReturn(Optional.of(employee));
        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(get("/api/employees/{id}", employeeId));

        // then - verify the output
        response.andExpect(status().isOk())
                .andDo(print())
                .andExpect(jsonPath("$.firstName", CoreMatchers.is(employee.getFirstName())))
                .andExpect(jsonPath("$.lastName", CoreMatchers.is(employee.getLastName())))
                .andExpect(jsonPath("$.email", CoreMatchers.is(employee.getEmail())));

    }

    //JUnit test for getEmployeeById method
    @DisplayName("JUnit test for getEmployeeById method (negative scenario)")
    @Test
    public  void givenEmployeeId_whenGetEmployeeId_thenReturnNegativeScenarioEmployeeObject() throws Exception {

        long employeeId = 1L;
        //given  - precondition or setup
        given(employeeService.findById(employeeId)).willReturn(Optional.empty());
        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(get("/api/employees/{id}", employeeId));

        // then - verify the output
        response.andExpect(status().isNotFound())
                .andDo(print());

    }

    //JUnit test for update employee REST API
    @DisplayName("JUnit test for update employee REST API (Positive scenario)")
    @Test
    public  void givenUpdatedEmployee_whenUpdateEmployee_thenReturnEmployeeObject() throws Exception {

        //given  - precondition or setup
        long id = 1L;
        given(employeeService.findById(id)).willReturn(Optional.of(employee));
        given(employeeService.updateEmployee(ArgumentMatchers.any(Employee.class)))
                .willAnswer(invocation -> invocation.getArgument(0));

        employee.setEmail("henry2@test.com");
        employee.setFirstName("test2");
        employee.setLastName("x3");

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(put("/api/employees/{id}", id)
                        .contentType(MediaType.APPLICATION_JSON)
                        .content(objectMapper.writeValueAsString(employee))
                    );

        // then - verify the output
        response.andExpect(status().isOk())
                .andDo(print())
                .andExpect(jsonPath("$.firstName", CoreMatchers.is(employee.getFirstName())))
                .andExpect(jsonPath("$.lastName", CoreMatchers.is(employee.getLastName())))
                .andExpect(jsonPath("$.email", CoreMatchers.is(employee.getEmail())));
    }

    //JUnit test for update employee REST API
    @DisplayName("JUnit test for update employee REST API (Negative scenario)")
    @Test
    public  void givenUpdatedEmployee_whenUpdateEmployee_thenReturnNegativeScenarioEmployee() throws Exception {

        //given  - precondition or setup
        long id = 1L;
        given(employeeService.findById(id)).willReturn(Optional.empty());
        given(employeeService.updateEmployee(ArgumentMatchers.any(Employee.class)))
                .willAnswer(invocation -> invocation.getArgument(0));

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(put("/api/employees/{id}", id)
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(employee))
        );

        // then - verify the output
        response.andExpect(status().isNotFound())
                .andDo(print());
    }

    //JUnit test for deleteEmployee
    @DisplayName("JUnit test for deleteEmployee")
    @Test
    public  void givenEmployeeId_whenDeleteEmployee_thenReturn200() throws Exception {

        long id = 1L;
        //given  - precondition or setup
        willDoNothing().given(employeeService).deleteEmployee(id);

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(delete("/api/employees/{id}", id));

        // then - verify the output
        response.andExpect(status().isOk())
                .andDo(print());

    }
}

7. Writing integration tests

Create a test class for integration testing:

package com.henry.demotesting.integration;


import com.fasterxml.jackson.databind.ObjectMapper;
import com.henry.demotesting.model.Employee;
import com.henry.demotesting.repository.EmployeeRepository;
import org.hamcrest.CoreMatchers;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;

import java.util.ArrayList;
import java.util.List;

import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@AutoConfigureMockMvc
public class EmployeeControllerITests {

    @Autowired
    private MockMvc mockMvc;

    @Autowired
    private EmployeeRepository employeeRepository;

    @Autowired
    private ObjectMapper objectMapper;

    private Employee employee;

    @BeforeEach
    void setup(){
        employeeRepository.deleteAll();

        employee = Employee.builder()
                .firstName("Henry")
                .lastName("x")
                .email("test@gmail.com")
                .build();
    }

    //JUnit test for createEmployee
    @DisplayName("JUnit test for create Employee")
    @Test
    public  void givenEmployeeObject_whenCreateEmployee_thenReturnSavedEmployee() throws Exception {

        //given  - precondition or setup

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(post("/api/employees")
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(employee))
        );

        // then - verify the output
        response.andDo(print())
                .andExpect(status().isCreated())
                .andExpect(jsonPath("$.firstName", CoreMatchers.is(employee.getFirstName())))
                .andExpect(jsonPath("$.lastName", CoreMatchers.is(employee.getLastName())))
                .andExpect(jsonPath("$.email", CoreMatchers.is(employee.getEmail())));
    }

    //JUnit test for getAllEmployees method
    @DisplayName("Junit test for getAllEmployees method")
    @Test
    public  void givenListOfEmployees_whenGetAllEmployees_thenReturnEmployeesList() throws Exception {

        var employee1 = Employee.builder()
                .firstName("Henry1")
                .lastName("x1")
                .email("test1@gmail.com")
                .build();

        //given  - precondition or setup
        List<Employee> listOfEmployees = new ArrayList<>();
        listOfEmployees.add(employee);
        listOfEmployees.add(employee1);
        employeeRepository.saveAll(listOfEmployees);
        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(get("/api/employees"));

        // then - verify the output
        response.andExpect(status().isOk())
                .andDo(print())
                .andExpect(jsonPath("$.size()"
                        , CoreMatchers.is(listOfEmployees.size())));

    }

    //JUnit test for getEmployeeById method
    @DisplayName("JUnit test for getEmployeeById method (positive scenario)")
    @Test
    public  void givenEmployeeId_whenGetEmployeeId_thenReturnEmployeeObject() throws Exception {

        //given  - precondition or setup
        var employee1 = employeeRepository.save(employee);
        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(get("/api/employees/{id}", employee1.getId()));

        // then - verify the output
        response.andExpect(status().isOk())
                .andDo(print())
                .andExpect(jsonPath("$.firstName", CoreMatchers.is(employee.getFirstName())))
                .andExpect(jsonPath("$.lastName", CoreMatchers.is(employee.getLastName())))
                .andExpect(jsonPath("$.email", CoreMatchers.is(employee.getEmail())));

    }

    //JUnit test for getEmployeeById method
    @DisplayName("JUnit test for getEmployeeById method (negative scenario)")
    @Test
    public  void givenEmployeeId_whenGetEmployeeId_thenReturnNegativeScenarioEmployeeObject() throws Exception {

        long employeeId = 1L;
        //given  - precondition or setup
        employeeRepository.save(employee);
        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(get("/api/employees/{id}", employeeId));

        // then - verify the output
        response.andExpect(status().isNotFound())
                .andDo(print());

    }

    //JUnit test for update employee REST API
    @DisplayName("JUnit test for update employee REST API (Positive scenario)")
    @Test
    public  void givenUpdatedEmployee_whenUpdateEmployee_thenReturnEmployeeObject() throws Exception {

        //given  - precondition or setup
        Employee employee1 = employeeRepository.save(employee);
        employee1.setEmail("henry2@test.com");
        employee1.setFirstName("test2");
        employee1.setLastName("x3");
        employeeRepository.save(employee1);

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(put("/api/employees/{id}", employee1.getId())
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(employee1))
        );

        // then - verify the output
        response.andExpect(status().isOk())
                .andDo(print())
                .andExpect(jsonPath("$.firstName", CoreMatchers.is(employee1.getFirstName())))
                .andExpect(jsonPath("$.lastName", CoreMatchers.is(employee1.getLastName())))
                .andExpect(jsonPath("$.email", CoreMatchers.is(employee1.getEmail())));
    }

    //JUnit test for update employee REST API
    @DisplayName("JUnit test for update employee REST API (Negative scenario)")
    @Test
    public  void givenUpdatedEmployee_whenUpdateEmployee_thenReturnNegativeScenarioEmployee() throws Exception {

        //given  - precondition or setup
        long id = 1L;
        employeeRepository.save(employee);

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(put("/api/employees/{id}", id)
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(employee))
        );

        // then - verify the output
        response.andExpect(status().isNotFound())
                .andDo(print());
    }

    //JUnit test for deleteEmployee
    @DisplayName("JUnit test for deleteEmployee")
    @Test
    public  void givenEmployeeId_whenDeleteEmployee_thenReturn200() throws Exception {

        //given  - precondition or setup
        var employee1 = employeeRepository.save(employee);

        //when - action or the behaviour that we are going test
        ResultActions response = mockMvc.perform(delete("/api/employees/{id}", employee1.getId()));

        // then - verify the output
        response.andExpect(status().isOk())
                .andDo(print());

    }
}

Conclusion

In this comprehensive guide, we've covered setting up a Spring Boot 3.2.5 application with a MySQL database using Docker Compose, and writing unit tests for the repository, service, and controller layers, as well as integration tests. This setup allows you to ensure that your application functions correctly across different layers and provides a solid foundation for further development and testing.

Source Code:

Here on GitHub.

References:

https://www.javaguides.net/

Canary Deployment on GKE with Terraform and Cloud Deploy

 Here's a detailed guide to setting up a Canary deployment on Google Cloud using Terraform and Cloud Deploy. This example demonstrates deploying a sample application image to a Google Kubernetes Engine (GKE) cluster, utilizing Canary deployment strategies to gradually roll out updates

Setting Up Terraform Configuration Files

provider.tf

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "5.18.0"
    }
  }
}

provider "google" {
  project = var.project_id
  region  = var.region
  zone    = var.zone
}

provider "google-beta" {
  project = var.project_id
  region  = var.region
  zone    = var.zone
}

variable.tf

variable "project_id" {
  default = ""
}

variable "region" {
  default = ""
}

variable "zone" {
  default = ""
}

variable "sec_region" {
  default = ""
}

variable "sec_zone" {
  default = ""
}

terraform.tfvars

project_id = <PROYECT-ID>
region     = "us-central1"
zone       = "us-central1-a"
sec_region = "us-west1"
sec_zone   = "us-west1-a"

Service Account Configuration

serviceaccount.tf

resource "google_service_account" "gke_service_account" {
  project      = var.project_id
  account_id   = "gke-service-account-id"
  display_name = "Service Account for Canary-deploy"
}

output "service_account_email" {
  value = google_service_account.gke_service_account.email
}

Network Configuration

network.tf

resource "google_compute_network" "nw1-vpc" {
  name                    = "nw1-vpc"
  auto_create_subnetworks = false
  mtu                     = 1460
}

resource "google_compute_subnetwork" "nw1-subnet1" {
  name                     = "nw1-vpc-sub1-${var.region}"
  network                  = google_compute_network.nw1-vpc.id
  ip_cidr_range            = "10.10.1.0/24"
  region                   = var.region
  private_ip_google_access = true
}

resource "google_compute_subnetwork" "nw1-subnet2" {
  name                     = "nw2-vpc-sub3-euro-west2"
  network                  = google_compute_network.nw1-vpc.id
  ip_cidr_range            = "10.10.2.0/24"
  region                   = "europe-west2"
  private_ip_google_access = true
}

resource "google_compute_firewall" "nw1-ssh-icmp-allow" {
  name    = "nw1-vpc-ssh-allow"
  network = google_compute_network.nw1-vpc.id
  allow {
    protocol = "icmp"
  }
  allow {
    protocol = "tcp"
    ports    = ["22"]
  }
  source_ranges = ["39.33.11.48/32"]
  target_tags   = ["nw1-vpc-ssh-allow"]
  priority      = 1000
}

resource "google_compute_firewall" "nw1-internal-allow" {
  name    = "nw1-vpc-internal-allow"
  network = google_compute_network.nw1-vpc.id
  allow {
    protocol = "icmp"
  }
  allow {
    protocol = "udp"
    ports    = ["0-65535"]
  }
  allow {
    protocol = "tcp"
    ports    = ["0-65535"]
  }
  source_ranges = ["10.10.0.0/16"]
  priority      = 1100
}

resource "google_compute_firewall" "nw1-iap-allow" {
  name    = "nw1-vpc-iap-allow"
  network = google_compute_network.nw1-vpc.id
  allow {
    protocol = "icmp"
  }
  allow {
    protocol = "tcp"
    ports    = ["0-65535"]
  }
  source_ranges = ["35.235.240.0/20"]
  priority      = 1200
}

resource "google_compute_address" "natpip" {
  name   = "ipv4-address"
  region = "europe-west2"
}

resource "google_compute_router" "router1" {
  name    = "nat-router1"
  region  = "europe-west2"
  network = google_compute_network.nw1-vpc.id
  bgp {
    asn = 64514
  }
}

resource "google_compute_router_nat" "nat1" {
  name                               = "natgw1"
  router                             = google_compute_router.router1.name
  region                             = "europe-west2"
  nat_ip_allocate_option             = "MANUAL_ONLY"
  nat_ips                            = [google_compute_address.natpip.self_link]
  source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"
  min_ports_per_vm                   = 256
  max_ports_per_vm                   = 512
  log_config {
    enable = true
    filter = "ERRORS_ONLY"
  }
}

resource "google_compute_global_address" "private_ip_address" {
  name          = google_compute_network.nw1-vpc.name
  purpose       = "VPC_PEERING"
  address_type  = "INTERNAL"
  prefix_length = 16
  network       = google_compute_network.nw1-vpc.name
}

resource "google_service_networking_connection" "private_vpc_connection" {
  network                 = google_compute_network.nw1-vpc.id
  service                 = "servicenetworking.googleapis.com"
  reserved_peering_ranges = [google_compute_global_address.private_ip_address.name]
}

IAM Configuration

iam.tf

resource "google_project_iam_member" "member-role" {
  depends_on = [google_service_account.gke_service_account]

  for_each = toset([
    "roles/iam.serviceAccountTokenCreator",
    "roles/clouddeploy.jobRunner",
    "roles/container.developer",
  ])
  role    = each.key
  project = var.project_id
  member  = "serviceAccount:${google_service_account.gke_service_account.email}"
}

GKE Cluster Configuration

main.tf

resource "google_container_cluster" "primary" {
  depends_on = [google_service_account.gke_service_account]

  name     = "canary-quickstart-cluster"
  location = var.region

  deletion_protection      = false
  remove_default_node_pool = true
  initial_node_count       = 1
  enable_shielded_nodes    = true

  node_config {
    service_account = google_service_account.gke_service_account.email
    oauth_scopes = [
      "https://www.googleapis.com/auth/cloud-platform"
    ]
    labels = {
      foo = "bar"
    }
    tags = ["foo", "bar"]
  }

  network    = google_compute_network.nw1-vpc.self_link
  subnetwork = google_compute_subnetwork.nw1-subnet1.self_link
}

resource "google_container_node_pool" "primary_preemptible_nodes" {
  depends_on = [
    google_container_cluster.primary,
    google_service_account.gke_service_account
  ]

  name       = "my-node-pool"
  location   = var.region
  cluster    = google_container_cluster.primary.name
  node_count = 1

  node_config {
    preemptible  = true
    machine_type = "e2-medium"
    service_account = google_service_account.gke_service_account.email
    oauth_scopes = [
      "https://www.googleapis.com/auth/cloud-platform"
    ]
  }
}

Running Terraform Commands

Initialize Terraform:

terraform init

Validate the configuration:

terraform validate

Apply the configuration:

terraform apply -auto-approve

Deploying the Application

Skaffold Configuration

Create a skaffold.yaml file:

apiVersion: skaffold/v4beta7
kind: Config
manifests:
  rawYaml:
  - kubernetes.yaml
deploy:
  kubectl: {}

Kubernetes Manifest

Create a kubernetes.yaml file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
  labels:
    app: my-app
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: nginx
        image: my-app-image
---
apiVersion: v1
kind: Service
metadata:
  name: my-service
  namespace: default
spec:
  selector:
    app: my-app
  ports:
    - protocol: TCP
      port: 80

Cloud Deploy Pipeline

Create a clouddeploy.yaml file:

apiVersion: deploy.cloud.google.com/v1
kind: DeliveryPipeline
metadata:
  name: my-canary-demo-app-1
description: main application pipeline
serialPipeline:
  stages:
  - targetId: prod
    profiles: []
    strategy:
      canary:
        runtimeConfig:
          kubernetes:
            serviceNetworking:
              service: "my-service"
              deployment: "my-deployment"
        canaryDeployment:
          percentages: [50]
          verify: false
---
apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: prod
description: prod GKE cluster
gke:
  cluster: projects/PROJECT_ID/locations/us-central1/clusters/canary-quickstart-cluster

Registering and Creating Releases

Register the pipeline:

gcloud deploy apply --file=clouddeploy.yaml --region=us-central1 --project=PROJECT_ID

Create the first release (skips canary phase):

gcloud deploy releases create test-release-001 \
  --project=PROJECT_ID \
  --region=us-central1 \
  --delivery-pipeline=my-canary-demo-app-1 \
  --images=my-app-image=gcr.io/google-containers/nginx@sha256:f49a843c290594dcf4d193535d1f4ba8af7d56cea2cf79d1e9554f077f1e7aaa

After the first release, the canary phase was skipped, and the rollout is waiting to start the "stable" phase, which deploys the application to 100%:

• In the pipeline visualization, click Advance to stable.
• When prompted, click Advance to confirm.

After a few minutes, the rollout is now in the "stable" phase, and the application is deployed to 100%.

Create the second release (executes canary deployment):

Because the first release skipped the canary phase skipped the canary phase, we'll now create another release, which this time does execute a canary deployment.

gcloud deploy releases create test-release-002 \
  --project=PROJECT_ID \
  --region=us-central1 \
  --delivery-pipeline=my-canary-demo-app-1 \
  --images=my-app-image=gcr.io/google-containers/nginx@sha256:f49a843c290594dcf4d193535d1f4ba8af7d56cea2cf79d1e9554f077f1e7aaa

When the first rollout phase finishes, the rollout is now in the canary phase:

This means that the application is now deployed to 50%. For serviceNetworking-based GKE, it's deployed to half of your pods. For Gateway API-based GKE and Cloud Run traffic is allocated to 50%.

Click Advance Rollout, then click Advance when prompted.

Monitoring Deployment

View and manage the deployment in the Google Cloud Console under the Cloud Deploy Delivery pipelines page. Follow the instructions to advance the rollout phases as necessary.

This comprehensive guide helps you set up a canary deployment pipeline on GKE using Terraform and Google Cloud Deploy, allowing you to roll out updates gradually and safely.

Source Code:

Here on GitHub.

References:

https://cloud.google.com/deploy/docs/deploy-app-canary#gke_4

https://cloud.google.com/config-connector/docs/how-to/install-upgrade-uninstall

Setting Up GitHub Actions with Google Cloud Workload Identity Federation

In this post, we'll walk you through integrating GitHub Actions with Google Cloud using Workload Identity Federation (WIF). This setup allows you to authenticate GitHub Actions to Google Cloud without using long-lived service account keys, improving security and manageability.

We'll use the following Terraform configurations and GitHub Actions workflow to achieve this:

Terraform Configuration

1. Configure Providers

Create a provider.tf file to configure the required Google providers.

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "4.0.0"
    }
  }
}

provider "google" {
  project = var.project_id
  region  = var.region
  zone    = var.zone
}

provider "google-beta" {
  project = var.project_id
  region  = var.region
  zone    = var.zone
} 

2. Define Variables

Create a variable.tf file to define the variables required for your project.

variable "project_id" {
  default = ""
}

variable "project_number" {
  default = ""
}

variable "region" {
  default = ""
}

variable "sec_region" {
  default = ""
}

variable "sec_zone" {
  default = ""
}

variable "zone" {
  default = ""
} 

3. Set Variable Values

Create a terraform.tfvars file to specify the values for the defined variables. Update your <PROJECT_ID> and <PROJECT_NO>

project_id = <PROJECT_ID>
project_number = <PROJECT_NO>
region     = "us-central1"
zone       = "us-central1-a"

sec_region = "us-west1"
sec_zone   = "us-west1-a"

4. Create a Service Account

Create a serviceaccount.tf file to define the service account and its roles.

resource "google_service_account" "github_svc" {
  project      = var.project_id
  account_id   = "gcp-github-access"
  display_name = "Service Account - github-svc"
}

resource "google_project_iam_member" "github-access" {
  for_each = toset([
    "roles/owner",
    "roles/iam.workloadIdentityPoolAdmin",
    "roles/iam.serviceAccountAdmin"
  ])
  role    = each.key
  project = var.project_id
  member  = "serviceAccount:${google_service_account.github_svc.email}"
} 

5. Enable Required APIs and Configure Workload Identity Federation

Create a wif.tf file to enable the necessary Google Cloud APIs and configure WIF. Update your <MY-USER-OR-COMPANY>/<MY-REPO-NAME>

resource "google_project_service" "wif_api" {
  for_each = toset([
    "iam.googleapis.com",
    "cloudresourcemanager.googleapis.com",
    "iamcredentials.googleapis.com",
    "sts.googleapis.com",
  ])

  service            = each.value
  disable_on_destroy = false
}

module "gh_oidc" {

  depends_on = [resource.google_service_account.github_svc]

  source            = "terraform-google-modules/github-actions-runners/google//modules/gh-oidc"
  version           = "v3.1.2" # Latest version
  project_id        = var.project_id
  pool_id           = "gh-identity-pool"
  pool_display_name = "Identity Pool"
  provider_id       = "gh-provider"
  sa_mapping = {
    (resource.google_service_account.github_svc.account_id) = {
      sa_name   = resource.google_service_account.github_svc.name
      attribute = "*"
    }
  }
}

resource "google_service_account_iam_binding" "github_svc_binding" {
  depends_on = [module.gh_oidc]
  service_account_id = google_service_account.github_svc.name
  role               = "roles/iam.workloadIdentityUser"
  members = [  "principalSet://iam.googleapis.com/projects/${var.project_number}/locations/global/workloadIdentityPools/gh-identity-pool/attribute.repository/<MY-USER-OR-COMPANY>/<MY-REPO-NAME>"
  ]
} 

Apply the Terraform Configuration

Once you have all the Terraform configuration files ready, follow these steps to apply the configuration:

• Initialize Terraform

terraform init

• Validate the Configuration

terraform validate

• Plan the Deployment

terraform plan

• Apply the Configuration

terraform apply -auto-approve

GitHub Actions Workflow

Create a workflow file in your GitHub repository under .github/workflows/wif.yml to define the GitHub Actions workflow. Update your <PROJECT_ID> and <PROJECT_NO> 

name: GitHub Actions WIF

on:
  workflow_dispatch:
    inputs:
      env:
        type: choice
        default: dv
        description: "Select the Environment"
        required: true
        options:
          - dv

env:
  PROJECT_ID: <PROJECT_ID>
  PROJECT_NO: <PROJECT_NO>
  POOL_ID: <POOL_ID>   #gh-identity-pool
  PROVIDER_ID: <PROVIDER_ID> #gh-provider
  
jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        
      - id: 'auth'
        name: 'Authenticate to Google Cloud'
        uses: 'google-github-actions/auth@v1'
        with:
          create_credentials_file: 'true'
          workload_identity_provider: 'projects/${{env.PROJECT_NO}}/locations/global/workloadIdentityPools/${{env.POOL_ID}}/providers/${{PROVIDER_ID}}'
          service_account: 'gcp-github-access@${{env.PROJECT_ID}}.iam.gserviceaccount.com'

      - name: 'Set up Cloud SDK'
        uses: 'google-github-actions/setup-gcloud@v1'
        with:
          version: '>= 363.0.0'

      - name: Run gcloud commands
        run: |
          gcloud config set project ${{env.PROJECT_ID}}
          gcloud version 

Summary

By following this setup, you enable secure authentication between GitHub Actions and Google Cloud using Workload Identity Federation. This eliminates the need for long-lived service account keys, enhancing the security of your CI/CD pipelines.

Source Code

Here on GitHub.

References

https://registry.terraform.io/modules/terraform-google-modules/github-actions-runners/google/latest/submodules/gh-oidc

Provisioning Cloud SQL with Private Service Connect Using Terraform & Accessing from Cloud Run with Spring Boot

In this post, we'll explore how to provision Cloud SQL instances with Private Service Connect (PSC) connectivity using Terraform and then access them from a Spring Boot application deployed on Google Cloud Run. We'll leverage Terraform to create the necessary infrastructure and configure the networking components. Then, we'll build and deploy a Spring Boot application that connects to this Cloud SQL instances using the appropriate methods.

Enabled APIs

The following APIs need to be enabled for this project:

• Cloud SQL API
• Cloud Run API
• Cloud Build API
• Artifact Registry API
• Cloud Logging API
• Serverless VPC Access API

Terraform Project Overview

The Terraform project sets up the following resources on Google Cloud Platform (GCP):

Virtual Private Cloud (VPC) Network and Subnets:

• A VPC network named nw1-vpc is created, along with two subnets (nw1-vpc-sub1-us-central1 and nw1-vpc-sub3-us-west1) in different regions.

Cloud SQL Instances:

• Private Service Connect (PSC) Instance: A Cloud SQL instance named psc-instance is created with Private Service Connect enabled, allowing secure access from Google Cloud services and resources.

Networking Components:

• Firewall rules are defined to control access to the VPC network.
• A NAT gateway is configured to allow instances in the VPC network to access the internet.

Service Account and IAM Roles:

• A service account named cloudsql-service-account-id is created and granted the necessary roles for accessing Cloud SQL instances.

Compare Direct VPC egress and VPC connectors

Cloud Run offers two methods for sending egress (outbound) traffic from a Cloud Run service or job to a VPC network:

VPC Connector:

• You can enable your Cloud Run service or job to send traffic to a VPC network by configuring a Serverless VPC Access connector. A VPC Connector named private-cloud-sql is provisioned for enabling Private Service Connect access from Google Cloud services like Cloud Run in network.tf.

Direct VPC egress:

• You can enable your Cloud Run service or job to send traffic to a VPC network by using Direct VPC egress with no Serverless VPC Access connector required.

Terraform Project

• Provider Configuration

In the provider.tf file, we define the required providers and configure the Google Cloud provider with the project ID, region, and zone:

provider "google" {
  project = var.project_id
  region  = var.region
  zone    = var.zone
}

The project_id, region, and zone variables are defined in the variables.tf file and assigned values in the terraform.tfvars file.

• Virtual Private Cloud (VPC) Network and Subnets

In the network.tf file, we create the VPC network and subnets:

resource "google_compute_network" "nw1-vpc" {
  project                 = var.project_id
  name                    = "nw1-vpc"
  auto_create_subnetworks = false
  mtu                     = 1460
}

resource "google_compute_subnetwork" "nw1-subnet1" {
  name                     = "nw1-vpc-sub1-${var.region}"
  network                  = google_compute_network.nw1-vpc.id
  ip_cidr_range            = "10.10.1.0/24"
  region                   = var.region
  private_ip_google_access = true
}

resource "google_compute_subnetwork" "nw1-subnet2" {
  name                     = "nw1-vpc-sub3-us-west1"
  network                  = google_compute_network.nw1-vpc.id
  ip_cidr_range            = "10.10.2.0/24"
  region                   = var.sec_region
  private_ip_google_access = true
}

• Private Service Connect (PSC) Instance

In the main.tf file, we create the Cloud SQL instances with Private Service Conect option:

resource "google_sql_database_instance" "psc_instance" {
  project          = var.project_id
  name             = "psc-instance"
  region           = var.region
  database_version = "POSTGRES_15"

  deletion_protection = false

  settings {
    tier = "db-f1-micro"
    ip_configuration {
      psc_config {
        psc_enabled               = true
        allowed_consumer_projects = ["<PROJECT_ID>"]
      }
      ipv4_enabled = false
    }
    availability_type = "REGIONAL"
  }
}

• Networking Components

resource "google_compute_global_address" "private_ip_address" {
  name          = google_compute_network.nw1-vpc.name
  purpose       = "VPC_PEERING"
  address_type  = "INTERNAL"
  prefix_length = 16
  network       = google_compute_network.nw1-vpc.name
}

resource "google_service_networking_connection" "private_vpc_connection" {
  network                 = google_compute_network.nw1-vpc.id
  service                 = "servicenetworking.googleapis.com"
  reserved_peering_ranges = [google_compute_global_address.private_ip_address.name]
}

• Service Account and IAM Roles

In the serviceaccount.tf file, we create the service account and assign necessary roles:

resource "google_service_account" "cloudsql_service_account" {
  project      = var.project_id
  account_id   = "cloudsql-service-account-id"
  display_name = "Service Account for Cloud SQL"
}

resource "google_project_iam_member" "member-role" {
  depends_on = [google_service_account.cloudsql_service_account]

  for_each = toset([
    "roles/cloudsql.client",
    "roles/cloudsql.editor",
    "roles/cloudsql.admin",
    "roles/secretmanager.secretAccessor",
    "roles/secretmanager.secretVersionManager",
    "roles/vpcaccess.serviceAgent"
  ])
  role    = each.key
  project = var.project_id
  member  = "serviceAccount:${google_service_account.cloudsql_service_account.email}"
}

• VPC Connector

In the network.tf file, we create the VPC Connector for Private Service Connect access:

#****************************Equivalent gcloud command
/* gcloud compute networks vpc-access connectors create private-cloud-sql  \
--region us-central1  \
--network nw1-vpc  \
--range "10.10.3.0/28"  \
--machine-type e2-micro  \
--project <PROJECT-ID>*/
resource "google_vpc_access_connector" "private-cloud-sql" {
  project       = var.project_id
  name          = "private-cloud-sql"
  region        = var.region
  network       = google_compute_network.nw1-vpc.id
  machine_type  = "e2-micro"
  ip_cidr_range = "10.10.3.0/28"
}

Private Service Connect Configuration

• Create a Private Service Connect endpoint

You can reserve an internal IP address for the Private Service Connect endpoint and create an endpoint with that address. To create the endpoint, you need the service attachment URI and the projects that are allowed for the instance.

To reserve an internal IP address for the Private Service Connect endpoint, use the gcloud compute addresses create command:

/*gcloud compute addresses create internal-address  \
--project=<PROJECT-ID>  \
--region=us-central1  \
--subnet=nw1-vpc-sub1-us-central1  \
--addresses=10.10.1.10*/

resource "google_compute_address" "internal_address" {
  project      = var.project_id
  name         = "internal-address"
  region       = var.region
  address_type = "INTERNAL"
  address      = "10.10.1.10"                               #"INTERNAL_IP_ADDRESS"
  subnetwork   = google_compute_subnetwork.nw1-subnet1.name 
} 

To create the Private Service Connect endpoint and point it to the Cloud SQL service attachment, use the gcloud compute forwarding-rules create command:

gcloud sql instances describe - displays configuration and metadata about a Cloud SQL instance

Get from this command:

• SERVICE_ATTACHMENT_URI (pscServiceAttachmentLink)

gcloud sql instances describe psc-instance --project PROJECT_ID 

gcloud compute forwarding-rules create psc-service-attachment-link \
--address=internal-address\
--project=PROJECT_ID \
--region=us-central1\
--network=nw1-vpc\
--target-service-attachment=SERVICE_ATTACHMENT_URI

• Configure a DNS managed zone and a DNS record

Cloud SQL doesn't create DNS records automatically. Instead, the instance lookup API response provides a suggested DNS name. We recommend that you create the DNS record in a private DNS zone in the corresponding VPC network. This provides a consistent way of using the Cloud SQL Auth Proxy to connect from different networks.

Get from this command:

• DNS Entry(dnsName)

gcloud sql instances describe psc-instance --project PROJECT_ID 

In the response, verify that the DNS name appears. This name has the following pattern: INSTANCE_UID.PROJECT_DNS_LABEL.REGION_NAME.sql.goog.. For example: 1a23b4cd5e67.1a2b345c6d27.us-central1.sql.goog.

To create a private DNS zone, use the gcloud dns managed-zones create command. This zone is associated with the VPC network that's used to connect to the Cloud SQL instance through the Private Service Connect endpoint.

gcloud dns managed-zones create cloud-sql-dns-zone\
--project=PROJECT_ID \
--description="DNS zone for the Cloud SQL instance"\
--dns-name=DNS_NAME \
--networks=nw1-vpc\
--visibility=private

Make the following replacements:

• ZONE_NAME: the name of the DNS zone
• PROJECT_ID: the ID or project number of the Google Cloud project that contains the zone
• DESCRIPTION: a description of the zone (for example, a DNS zone for the Cloud SQL instance)
• DNS_NAME: the DNS name for the zone, such as REGION_NAME.sql.goog. (where REGION_NAME is the region name for the zone)
• NETWORK_NAME: the name of the VPC network

After you create the Private Service Connect endpoint, to create a DNS record in the zone, use the gcloud dns record-sets create command:

gcloud dns record-sets create DNS_NAME \
--project=PROJECT_ID \
--type=A\
--rrdatas=10.10.1.10\
--zone=cloud-sql-dns-zone

Make the following replacements:

• DNS_NAME: the DNS name that you retrieved earlier in this procedure.
• RRSET_TYPE: the resource record type of the DNS record set (for example, A).
• RR_DATA: the IP address allocated for the Private Service Connect endpoint (for example, 198.51.100.5). You can also enter multiple values such as rrdata1 rrdata2 rrdata3 (for example, 10.1.2.3 10.2.3.4 10.3.4.5).

Spring Boot Application

• Data Sources Configuration

In the application.yaml file, we configure the data sources for Private Service connect Cloud SQL instances:

spring:
  jpa:
    defer-datasource-initialization: true
  sql:
    init:
      mode: always
  datasource:
    psc:
      url: jdbc:postgresql:///
      database: my-database3
      cloudSqlInstance: <PROJECT-ID>:<REGION>:psc-instance
      username: <user>
      password: <passoword>
      ipTypes: PSC
      socketFactory: com.google.cloud.sql.postgres.SocketFactory
      driverClassName: org.postgresql.Driver 

• Database Configuration Classes

ackage com.henry.democloudsql.configuration;


import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.JpaVendorAdapter;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter;
import org.springframework.transaction.PlatformTransactionManager;

import javax.naming.NamingException;
import javax.sql.DataSource;
import java.util.Properties;

@Configuration
@EnableJpaRepositories(
        basePackages = "com.henry.democloudsql.repository",
        entityManagerFactoryRef = "pscEntityManager",
        transactionManagerRef = "pscTransactionManager"
)
public class PSCpostgresConfig {

    @Value("${spring.datasource.psc.url}")
    private  String url;
    @Value("${spring.datasource.psc.database}")
    private String database;
    @Value("${spring.datasource.psc.cloudSqlInstance}")
    private  String cloudSqlInstance;
    @Value("${spring.datasource.psc.username}")
    private String username;

    @Value("${spring.datasource.psc.password}")
    private String password;
    @Value("${spring.datasource.psc.ipTypes}")
    private String ipTypes;
    @Value("${spring.datasource.psc.socketFactory}")
    private String socketFactory;
    @Value("${spring.datasource.psc.driverClassName}")
    private String driverClassName;
    @Bean
    @Primary
    public LocalContainerEntityManagerFactoryBean pscEntityManager()
            throws NamingException {
        LocalContainerEntityManagerFactoryBean em
                = new LocalContainerEntityManagerFactoryBean();
        em.setDataSource(pscDataSource());
        em.setPackagesToScan("com.henry.democloudsql.model");

        JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
        em.setJpaVendorAdapter(vendorAdapter);
        em.setJpaProperties(pscHibernateProperties());

        return em;
    }
    @Bean
    @Primary
    public DataSource pscDataSource() throws IllegalArgumentException {
        HikariConfig config = new HikariConfig();

        config.setJdbcUrl(String.format(url + "%s", database));
        config.setUsername(username);
        config.setPassword(password);

        config.addDataSourceProperty("socketFactory", socketFactory);
        config.addDataSourceProperty("cloudSqlInstance", cloudSqlInstance);

        config.addDataSourceProperty("ipTypes", ipTypes);

        config.setMaximumPoolSize(5);
        config.setMinimumIdle(5);
        config.setConnectionTimeout(10000);
        config.setIdleTimeout(600000);
        config.setMaxLifetime(1800000);

        return new HikariDataSource(config);
    }
    private Properties pscHibernateProperties() {
        Properties properties = new Properties();
        return properties;
    }
    @Bean
    @Primary
    public PlatformTransactionManager pscTransactionManager() throws NamingException {
        final JpaTransactionManager transactionManager = new JpaTransactionManager();
        transactionManager.setEntityManagerFactory(pscEntityManager().getObject());
        return transactionManager;
    }
} 

• Entity Models

package com.henry.democloudsql.model;

import jakarta.persistence.*;
import lombok.*;

import java.math.BigDecimal;

@Entity
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
@Table(name = "table3")
public class Table3 {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "product")
    private String product;

    @Column(name = "price")
    private BigDecimal price;
} 

• Repositories

package com.henry.democloudsql.repository;

import com.henry.democloudsql.model.Table3;
import org.springframework.data.repository.CrudRepository;

public interface Table3Repository extends CrudRepository<Table3, Long> {
} 

• Services

package com.henry.democloudsql.service;

public sealed  interface DefaultService<T, G> permits Table3ServiceImpl {
    T save(T obj);
    Iterable<T>  findAll();
    T findById(G id);
} 

package com.henry.democloudsql.service;

import com.henry.democloudsql.repository.Table3Repository;
import org.springframework.stereotype.Service;

@Service
public final class Table3ServiceImpl implements DefaultService {

    private  final Table3Repository table3Repository;

    public Table3ServiceImpl(Table3Repository table3Repository) {
        this.table3Repository = table3Repository;
    }

    @Override
    public Object save(Object obj) {
        return null;
    }

    @Override
    public Iterable findAll() {
        return table3Repository.findAll();
    }

    @Override
    public Object findById(Object id) {
        return null;
    }
} 

• Controllers

package com.henry.democloudsql.controller;


import com.henry.democloudsql.model.Table3;
import com.henry.democloudsql.service.DefaultService;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/v3")
public class Table3Controller {

    private final DefaultService<Table3, Long> defaultService;

    public Table3Controller(DefaultService<Table3, Long> defaultService) {
        this.defaultService = defaultService;
    }

    @GetMapping
    public Iterable<Table3> findAll(){
        return defaultService.findAll();
    }
} 

1. Dockerization

The Dockerfile is used to build the Docker image for the Spring Boot application:

FROM openjdk:17-jdk-slim
WORKDIR /app
COPY target/demo-cloudsql.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]

This Dockerfile uses the openjdk:17-jdk-slim base image, sets the working directory to /app, copies the built Spring Boot JAR file (demo-cloudsql.jar) into the container, and specifies the entrypoint to run the JAR file.

After creating the Dockerfile, you can build the Docker image locally using the following command:

Additional Details

• Executing Terraform Commands:

Before deploying the Spring Boot application, run the following Terraform commands to provision the infrastructure:

terraform init

terraform validate

terraform apply -auto-approve

• Building and Deploying Spring Boot Application:

After modifying MY_PROJECT_ID in application.yml on Spring Boot App, run:

mvn clean install

After clean install, you can build the Docker image locally using the following command:

docker build -t quickstart-springboot:1.0.1 .

This command builds the Docker image with the tag quickstart-springboot:1.0.1 using the Dockerfile in the current directory.

Deployment and Integration

• Artifact Repository was created by tf project

resource "google_artifact_registry_repository" "my-repo" {
  location      = var.region
  repository_id = "my-repo"
  description   = "example docker repository"
  format        = "DOCKER"
}

• Push Docker Image to Artifact Registry

To push the Docker image to the Artifact Registry, you first need to tag it with the appropriate URL:

docker tag quickstart-springboot:1.0.1 us-central1-docker.pkg.dev/MY_PROJECT_ID/my-repo/quickstart-springboot:1.0.1

Replace MY_PROJECT_ID with your actual GCP project ID.

Then, push the tagged image to the Artifact Registry:

docker push us-central1-docker.pkg.dev/MY_PROJECT_ID/my-repo/quickstart-springboot:1.0.1

Deploy to Cloud Run

Deploy the Spring Boot application to Google Cloud Run using the gcloud command:

With VPC Connector:

gcloud run deploy springboot-run-psc-vpc-connector \
  --image us-central1-docker.pkg.dev/MY_PROJECT_ID/my-repo/quickstart-springboot:1.0.1 \
  --region=us-central1 \
  --allow-unauthenticated \
  --service-account=cloudsql-service-account-id@terraform-workspace-413615.iam.gserviceaccount.com \
  --vpc-connector private-cloud-sql

With Direct VPC egress:

gcloud beta run deploy springboot-run-psc-direct-vpc-egress  \
--image=us-central1-docker.pkg.dev/MY_PROJECT_ID/my-repo/quickstart-springboot:1.0.1 \
--allow-unauthenticated  \
--service-account=cloudsql-service-account-id@terraform-workspace-413615.iam.gserviceaccount.com  \
--network=nw1-vpc  \
--subnet=nw1-vpc-sub1-us-central1  \
--vpc-egress=all-traffic  \
--region=us-central1  \
--project=MY_PROJECT_ID 

This command deploys the Spring Boot application to Cloud Run, using the Docker image from the Artifact Registry. It also specifies the service account created by Terraform (cloudsql-service-account-id@terraform-workspace-413615.iam.gserviceaccount.com)

TEST

Source Code

Here on GitHub.

References

https://cloud.google.com/vpc/docs/private-service-connect

https://cloud.google.com/run/docs/configuring/connecting-vpc

https://cloud.google.com/run/docs/configuring/vpc-direct-vpc

https://cloud.google.com/run/docs/configuring/vpc-connectors#console