Saturday, July 16, 2022

Java Streams: Read files in Parallel, Sequential and Per Chunk


Parallel processing a problem in multiple cores each process running in a separate thread. however, consider the overhead. In this example it is faster to get the paths names files  in parallel than sequential, however it is faster or equal to read the content of each file by sequential than parallel. So the parallel stream will be used in specific cases.  for example when calling a function or stored procedure from databases at the same time with different parameters or something like that.

Read files in Parallel.

Parallel computing involves dividing a problem into subproblems, solving those problems simultaneously (in parallel, with each subproblem running in a separate thread), and then combining the results of the solutions to the subproblems. 

normalize() :  Return current path and remove redundant name elements.
isRegularFile():  Return true if regular file, and false is not a regular file for example a  directory.

import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Collectors;

public class Demo2 {

public static void main(String[] args) {

long startTime = System.currentTimeMillis();

try {
String directory = "C:\\demo";
var files = Files.walk(Paths.get(directory))
.parallel().map(Path::normalize)
.filter(Files::isRegularFile)
.filter(path -> path.getFileName().toString().endsWith(".txt"))
.collect(Collectors.toList());

files.stream().parallel().forEach(item -> {
System.out.println(item + " " + Thread.currentThread().getName());
try {
Files.lines(Paths.get(item.toString())).parallel().forEach(val -> {
System.out.println("value -> " + val);
});
} catch (IOException e) {
throw new RuntimeException(e);
}
});

} catch (IOException e) {
throw new RuntimeException(e);
}

long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime - startTime;
System.out.println("millisecond "+elapsedTime);
}
}

Result this:

C:\demo\lines1.txt ForkJoinPool.commonPool-worker-1
C:\demo\lines3.txt ForkJoinPool.commonPool-worker-2
C:\demo\lines2.txt main
value -> third file2
value -> first file1
value -> second file2
value -> third file1
value -> first file2
value -> second file1
millisecond 35



Read files in Sequential.

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Collectors;

public class Demo3 {

public static void main(String[] args) {

long startTime = System.currentTimeMillis();

try {
String directory = "C:\\demo";
var files = Files.walk(Paths.get(directory))
.map(Path::normalize)
.filter(Files::isRegularFile)
.filter(path -> path.getFileName().toString().endsWith(".txt"))
.collect(Collectors.toList());

files.stream().forEach(item -> {
System.out.println(item + " " + Thread.currentThread().getName());
try {
Files.lines(Paths.get(item.toString())).forEach(val -> {
System.out.println("value -> " + val);
});
} catch (IOException e) {
throw new RuntimeException(e);
}
});

} catch (Exception e) {
throw new RuntimeException(e);
}

long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime - startTime;
System.out.println("millisecond " + elapsedTime);
}
}

Result this:

C:\demo\lines1.txt main
value -> first file1
value -> first file2
C:\demo\lines2.txt main
value -> second file1
value -> second file2
C:\demo\lines3.txt main
value -> third file1
value -> third file2
millisecond 38



Read files Per Chunk.

It used AtomicInteger for counting the number lines in the file. Files.lines used by threads concurrently.

The file contains 10 rows then in this example  read the file per chunk 3 rows. 


import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;

public class Demo1 {

public static void main(String[] args) {

try {

final AtomicInteger c = new AtomicInteger();
final AtomicInteger count = new AtomicInteger(1);

String fileName = "C:\\demo\\lines1.txt";
Files.lines(Paths.get(fileName))
.collect(Collectors.groupingBy(e -> c.getAndIncrement() / 3))//per 3
.forEach((fi, item) -> {
System.out.println("Chunk " + fi + " List " + item);
for (String line : item) {
System.out.println(" Number line -> " + count.getAndIncrement() + " -> " + line);
}

});

} catch (IOException e) {
e.printStackTrace();
}
}
}

Result this:

Chunk 0 List [first file1, first file2, first file3]
 Number line -> 1 -> first file1
 Number line -> 2 -> first file2
 Number line -> 3 -> first file3
Chunk 1 List [first file4, first file5, first file6]
 Number line -> 4 -> first file4
 Number line -> 5 -> first file5
 Number line -> 6 -> first file6
Chunk 2 List [first file7, first file8, first file9]
 Number line -> 7 -> first file7
 Number line -> 8 -> first file8
 Number line -> 9 -> first file9
Chunk 3 List [first file10]
 Number line -> 10 -> first file10











References:
https://docs.oracle.com/javase/tutorial/collections/streams/parallelism.html
https://www.baeldung.com/java-when-to-use-parallel-stream
https://mkyong.com/java8/java-8-parallel-streams-examples/
https://stackoverflow.com/questions/27583623/is-there-an-elegant-way-to-process-a-stream-in-chunks
https://howtodoinjava.com/java/multi-threading/atomicinteger-example/
https://www.geeksforgeeks.org/path-normalize-method-in-java-with-examples/
https://www.logicbig.com/how-to/code-snippets/jcode-java-io-files-isregularfile.html










































































































































Tuesday, July 5, 2022

Keycloak 17 , LDAP, Angular 14 And Spring Boot 2

 

Keycloak


Keycloak is an open source identity and access management for modern applications and services, no need to deal with storing users or authenticating users. it's all available out of the box.

The Documentation keycloak mentioned the following with respect LDAP and Active Directory:  Keycloak comes with a built-in LDAP/AD provider. It is possible to federate multiple different LDAP servers in the same Keycloak realm. You can map LDAP user attributes into the Keycloak common user model. By default, it maps username, email, first name, and last name, but you are free to configure additional mappings.

OpenID Connect

OpenID Connect additional is a simple identity layer on top of the OAuth 2.0 protocol. It allows Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.




Downloading and installing docker-test-openldap by Docker


For Install docker-test-openldap you just need the next command: it will check the users available on GitHub repository.

docker run --rm -p 10389:10389 -p 10636:10636 rroemhild/test-openldap


Test connection to LDAP on Apache Directory Studio















Downloading and installing keycloack by Docker


For Install keycloak by docker you just need the next command: I used version 17.0.1

docker run -p 8080:8080 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin quay.io/keycloak/keycloak:17.0.1 start-dev
This will start Keycloak exposed on the local port 8080. It will also create an initial admin user with username admin and password admin.

Setting Up a keycloack server.


Open a browser  http://localhost:8080 or http://127.0.0.1:8080 result this.





Creating a Realm

For default the console opens up Master realm, Let's navigate left corner Add realm button.













For this example I created realm demo. we'll click the button create.


Creating a Client









Add a new client with the name spring-boot-keycloak with openid-connect. click button save.












Add Valid Redirect URIs.



Add LDAP


Click on User Federation -> ldap:













Ldap Parameters:

Edit Mode: READ_ONLY
Vendor: Active Directory
LDAP attribute name for username: uid
RDN LDAP attribute: cn
UUID LDAP attribute: cn
User Object Classes: top, person, organizationalPerson, inetOrgPerson
Connection URL: ldap://YOUR_IP:10389
Users DN: ou=people,dc=planetexpress,dc=com
Search Scope: One level
Bind Type: none
























Click and Save and Synchronize all user.

Go to Manage -> Users and check all users:



























Creating a Spring Boot Application


In this examples we're integration Spring Boot, keycloack and Ldap user federation.

  • Configuration Spring boot, Spring Security and keycloack.
  • Spring Controllers  

Spring Boot Rest  API 

This is API

MethodsURLs
GET/users

Technology

  • Java 17 (Zulu)
  • Spring Boot 2.7.1
  • Maven 
  • IntelliJ IDEA 
  • Docker
  • Test-openldap

Project Structure



























Configuration Spring Boot Keycloak project 


pom.xml


<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.1</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.henry</groupId>
<artifactId>demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo</name>
<description>Demo project for Spring Boot Keycloak, ldap</description>

<properties>
<java.version>17</java.version>
<keycloak-adapter-bom.version>17.0.1</keycloak-adapter-bom.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.keycloak.bom</groupId>
<artifactId>keycloak-adapter-bom</artifactId>
<version>${keycloak-adapter-bom.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<!-- Exclude the Tomcat dependency -->
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Use Jetty instead -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<scope>runtime</scope>
</dependency>

<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>${project.parent.version}</version>
</plugin>
</plugins>
</build>

</project>

application.yml


server:
port: 8081
keycloak:
auth-server-url: http://127.0.0.1:8080
realm: demo
resource: spring-boot-keycloak
public-client: true
bearer-only: true
principal-attribute: preferred_username

Class Security configuration


I suggest to you, we 'll be reading Spring Boot Adapters, I got from Spring Boot Adapters the configuration class.

import org.keycloak.adapters.springboot.KeycloakSpringBootConfigResolver;
import org.keycloak.adapters.springsecurity.KeycloakConfiguration;
import org.keycloak.adapters.springsecurity.authentication.KeycloakAuthenticationProvider;
import org.keycloak.adapters.springsecurity.config.KeycloakWebSecurityConfigurerAdapter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Import;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.method.configuration.EnableGlobalMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.core.authority.mapping.SimpleAuthorityMapper;
import org.springframework.security.core.session.SessionRegistryImpl;
import org.springframework.security.web.authentication.session.RegisterSessionAuthenticationStrategy;
import org.springframework.security.web.authentication.session.SessionAuthenticationStrategy;
import org.springframework.web.cors.CorsConfiguration;
import org.springframework.web.cors.CorsConfigurationSource;
import org.springframework.web.cors.UrlBasedCorsConfigurationSource;

@KeycloakConfiguration
@Import(KeycloakSpringBootConfigResolver.class)
@EnableGlobalMethodSecurity(jsr250Enabled = true)//so i need the roles exist environment, @RolesAllowed
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter
{


/**
* Registers the KeycloakAuthenticationProvider with the authentication manager.
*/
@Autowired
public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception {

KeycloakAuthenticationProvider keycloakAuthenticationProvider = new KeycloakAuthenticationProvider();
keycloakAuthenticationProvider.setGrantedAuthoritiesMapper(new SimpleAuthorityMapper());
auth.authenticationProvider(keycloakAuthenticationProvider);
}

/**
* Defines the session authentication strategy.
*/
@Bean
@Override
protected SessionAuthenticationStrategy sessionAuthenticationStrategy() {
return new RegisterSessionAuthenticationStrategy(new SessionRegistryImpl());
}

@Override
protected void configure(HttpSecurity http) throws Exception
{
super.configure(http);
http.cors().and()
.authorizeRequests()
.anyRequest().fullyAuthenticated();
http.csrf().disable();

}

@Bean
CorsConfigurationSource corsConfigurationSource() {
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
source.registerCorsConfiguration("/**", new CorsConfiguration().applyPermitDefaultValues());
return source;
}
}

Spring Rest APIs Controller


import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.Collections;
import java.util.Map;

@RestController

public class HelloWorldController {

@GetMapping("/users")
public Map<String, String> hello(){
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
String currentPrincipalName = authentication.getName();
return Collections.singletonMap("response", "Hello "+currentPrincipalName);

}
}


Run & Test


Run Spring Boot application with command: mvn spring-boot:run. by console, IntelliJ etc.

Keycloak provides a REST API for generating and refreshing access tokens. 

1. We need to acquire an access token from Keycloak from this url. 

  • 127.0.0.1 or localhost .
  • Port: 8080 .
  • realms Url defined by keycloak.
  • demo The name of realm.
  • protocol/openid-connect/token The url defined by keycloak.

 http://127.0.0.1:8080/realms/demo/protocol/openid-connect/token

2. Configurate on Postman.


Without token:
GET: http://127.0.0.1:8081/users




With token:
Parameters:

GET: http://127.0.0.1:8081/users
Tab Authorizacion: for more users it will check of Test-openldap on GitHub.
       Type: OAuth 2.0
       Grant Type: Password Credentials
       Access Token URL: http://127.0.0.1:8080/realms/demo/protocol/openid-connect/token
       Client ID: spring-boot-keycloak
       Username: zoidberg
       Password: zoidberg
       Client Authentication: Send client credentials in body











Click on Get New Acces token.

















Click Proceed and next to Use Token.














Creating  Angular Application


In this examples we're integration Angular, keycloack, Ldap user federation and Spring Boot.

  • Configuration Angular and keycloack.
  • Components
  • Services


Angular Rest  API 

This is API

MethodsURLs
GET/users

Technology

  • Visual Studio Code 1.68.1
  • Node 16
  • Npm
  • Angular Cli 14
  • Angular Material 14
  • sweetalert2

Project Structure






























Configuration Angular Keycloak project 


Folder: 

redirect: When the app is up always start in this component. 


    

  






    login: provided ts, scss and html file.






    






    pages: provided title header in all pages and all components to configure routing file. 





   














     services: provided services, interceptors, guards, consuming web services etc.




















 environments: (environment.ts) provided the endpoints for keycloak and spring boot.

// This file can be replaced during build by using the `fileReplacements` array.
// `ng build` replaces `environment.ts` with `environment.prod.ts`.
// The list of file replacements can be found in `angular.json`.

export const environment = {
  production: false,
  keycloakEndpoint: "http://127.0.0.1:8080/realms/demo/protocol/openid-connect/token",
  wsEndpoint: "http://127.0.0.1:8081"
};

/*
 * For easier debugging in development mode, you can import the following file
 * to ignore zone related error stack frames such as `zone.run`, `zoneDelegate.invokeTask`.
 *
 * This import should be commented out in production mode because it will have a negative impact
 * on performance if an error is thrown.
 */
// import 'zone.js/plugins/zone-error';  // Included with Angular CLI.

 
 

Run & Test


Run Angular application with command: ng serve










Go to http://localhost:4200/
Set credentials for users from LDAP User federation from Keycloak.

username: zoidberg
password: zoidberg

















Click on Login

If login is ok.  Show the name from Keycloak and username from Spring Boot project.

Get and set values from login html (login.component.ts).

 const body = new HttpParams()
      .set("username", this.loginForm.value.username!)
      .set("password", this.loginForm.value.password!)
      .set("grant_type", "password")
      .set("client_id", "spring-boot-keycloak")
      .set("client_secret", "")
      .set("scope", "openid");

Call WS KeycloakEndopoint (auth.service.ts).

async currentloginid(payMod: any) {
    //changes this url by your realms
    try {
      const response = await fetch(environment.keycloakEndpoint, {
        method: 'post',
        headers: {
          'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8'
        },
        body: payMod
      });
      const data = await response.json();
      return data;
    } catch (error) {
      console.log('Request failed', error);
    }
  }


Call WS Endpoint Spring Boot Project (content.component.ts).

ngOnInit(): void {
    this.name = this.authService.getUser();
    let url = `${environment.wsEndpoint}/users`;
    this._client.getUserFromBackend(url).subscribe((data) => {
      this.username = data.response;
    });
  }




Go to inspect: Righ click on page -> Inspect or f12.

Tab -> Application -> Storage ->Session Storage 

Show the token.







For will check if token is valid go to jwt.io and copy and paste token.


Source Code


Here on GitHub:




References.

https://github.com/rroemhild/docker-test-openldap
https://jwt.io/
https://www.keycloak.org/getting-started/getting-started-docker
https://www.keycloak.org/
https://material.angular.io/guide/schematics
https://angular.io/









🚀 Spring Boot 3.5 → 4.0.3 Migration Summary

Import Changes, API Adjustments & Compatibility Results Migration summary from Spring Boot 3.5 → 4.0, including breaking changes, depend...