Improving Data Search with Elasticsearch in Spring Boot Applications

Elasticsearch has become one of the top choices for addressing complex data search and analysis needs. Known for its speed and scalability, Elasticsearch offers excellent solutions for a variety of use cases. This article will discuss why Elasticsearch is a strong choice and best practices for implementation in various contexts.

Why Elasticsearch?

Elasticsearch is a distributed search and analysis engine built on Apache Lucene. Some reasons why we should consider Elasticsearch:

1. High Performance: Elasticsearch is designed to provide high performance, with the ability to handle large amounts of data and deliver search results instantly.
2. Horizontal Scalability: With its horizontal scalability capabilities, Elasticsearch can be easily scaled up by adding more nodes.
3. Full-Text Search: Elasticsearch provides powerful full-text search capabilities and supports complex queries.
4. Data Analysis Capabilities: In addition to search, Elasticsearch can be used for data analysis, visualization, and monitoring.
5. Support for Flexible Data Structures: Data can be modeled very flexibly in Elasticsearch, allowing customization to a wide range of data types.

In the implementation of Elasticsearch, best practices need to be applied in order to obtain optimal performance. Index planning is the first step, where the selection of settings such as the number of shards (divisible parts of the index) and replicas (duplication of shards) is tailored to the search and analysis needs. Optimal mapping is of key importance, where planning and determining the index structure must match the search requirements. Query and filter processes must be used wisely to ensure accurate and efficient search results. Active monitoring is essential, allowing the tracking of Elasticsearch performance, and regular maintenance such as segment merging and index optimization is required to maintain system performance. In use, Elasticsearch works by distributing data into shards, which can be subdivided into replicas to increase data redundancy and availability. Mapping is used to determine how data is indexed and stored, ensuring a structure that fits the needs. Indexes are data repositories organized by Elasticsearch to facilitate search and analysis. With this understanding, users can design and manage Elasticsearch systems effectively.

Before trying to apply Springboot with ElasticSearch I used the project that I had made in the previous article, you can read it here.

1. Configure Dependencies

To start the Elasticsearch integration in a Spring application, the first step is to add the required dependencies into the pom.xml file. This can be achieved by including dependencies for Spring Data Elasticsearch. Make sure to customize the version of the dependencies as per the project requirements.

<!-- Spring Data ElasticSearch -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

After adding these dependencies, don’t forget to add connection config spring

# ElasticSearch
spring.data.elasticsearch.cluster-nodes=localhost:9200
spring.data.elasticsearch.cluster-name=product_index

2. Model Entity Modification

Next, modifications to the model entities need to be made to suit the Elasticsearch integration. On the Product model class, add an @Document annotation with an indexName property. This annotation informs Spring Data Elasticsearch that this model class will be indexed in an index named “product_index”.

@Document(indexName = "product_index") //index for elasticsearch
public class Product implements Serializable {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;

    private String name;
    private int price;
    private String description;
}

3. Repository Configuration and Conflict Avoidance

When using Elasticsearch together with a JPA repository, additional configuration is required to avoid inter-repository conflicts. Create a new configuration class, in this example called JpaConfig, and add @Configuration and @EnableJpaRepositories annotations specifying different base packages from the JPA repository.

@Configuration
@EnableJpaRepositories(basePackages = "com.callmezydd.rediscrud.repository.jpa")
public class JpaConfig {
}

After that, in the Elasticsearch repository (ProductElasticRepo), add the @Repository annotation to tell Spring that this is a repository component.

@Repository
public interface ProductElasticRepo extends ElasticsearchRepository<Product, Long> {
    List<Product> findByNameOrDescription(String name, String description);
}

With these steps, your Spring project has been configured to integrate and interact with Elasticsearch simultaneously with the JPA repository.

4. Adding New Service for Search

In the previous CRUD development, a new service was added to accommodate specialized search operations using Elasticsearch. In the getAllProductsFromElasticSearch method, this service utilizes the @Cacheable annotation to cache the Elasticsearch query results. Using the Elasticsearch repository (ProductElasticRepo), this service searches for products by name or description that match the received query parameters. An information log is used to monitor the query process and results.

@Cacheable(value = "allProductES", key = "'products'")
public List<Product> getAllProductsFromElasticSearch(String query) {
    logger.info("Fetching all products from Elasticsearch with query: " + query);
    List<Product> result = productElasticRepo.findByNameOrDescription(query, query);
    logger.info("Query result: " + result);
    return result;
}

In other CRUD operations such as saveProduct, update, and delete, there are customizations to interact with Elasticsearch. For example, in the saveProduct method, the data is saved using the JPA repository (productRepo) as usual, and then the data is also saved in Elasticsearch using the Elasticsearch repository (productElasticRepo).

@CacheEvict(value = {"allProduct", "allProductES"}, key = "'products'")
public Product saveProduct(Product product){
    logger.info("Saving product: " + product.getName() + " to database");
    Product savedProduct = productRepo.save(product);
    //save product to elastic
    productElasticRepo.save(savedProduct);
    logger.info("Product saved successfully with ID: " + savedProduct.getId());
    return savedProduct;
}

5. Adding New Endpoint

As a final step, a new endpoint is added to the controller class to utilize the custom search service that has been created. In this example, the getAllProductsFromElasticSearch endpoint uses the @GetMapping annotation with path variables, so that the query becomes more dynamic and can be customized based on the product name or description.

// get all from Elasticsearch
@GetMapping("/es/{query}")
public List<Product> getAllProductsFromElasticSearch(@PathVariable String query) {
    return productService.getAllProductsFromElasticSearch(query);
}

As such, the integration of Elasticsearch in this Spring CRUD project allows developers to utilize specialized search services as well as traditional CRUD operations. The entire workflow is presented in detail and can be implemented in various application development contexts.

6. Running Redis, Elasticsearch, and Spring Servers

In the last stage, to run the Redis server, Elasticsearch, and Spring applications, Docker Compose is used. In the docker-compose.yaml file, there is a configuration for each service that will be run.

version: "3.8"

services:
  redis:
    image: redis:7.2.3
    container_name: redis-container
    ports:
      - "6379:6379"

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3
    container_name: elasticsearch-container
    environment:
      - discovery.type=single-node
      - cluster.name=product_index
    ports:
      - "9200:9200"

  spring-app:
    build: .
    depends_on:
      - redis
      - elasticsearch
    ports:
      - "8080:8080"
    command: "./wait-for-it.sh elasticsearch:9200 -- mvn spring-boot:run"

The environment configuration includes the discovery.type=single-node option which indicates that the Elasticsearch node is running in single-node mode. In addition, a cluster name is given with cluster.name=product_index. Port 9200 is used to communicate with Elasticsearch.

The Spring application is set up in Docker Compose by building an image using a Dockerfile (build: .). The service has dependencies on Redis and Elasticsearch (depends_on). The exposed external port is 8080, and before running the Spring application with the mvn spring-boot:run command, a wait is performed until the Elasticsearch service is considered ready using the wait-for-it.sh script. Once the configuration is complete, the complete infrastructure, including Redis, Elasticsearch, and the Spring application, can be run simultaneously with the docker-compose up -d command.

To test the functionality, you can add new data through Postman using the /products endpoint with the POST method. After adding the data, check the results in Elasticsearch via the Elasticsearch host URL, which is http://localhost:9200/product_index/_search?q=*.

Add Data to product_index

If successful, the new data should appear. Next, you can test the new endpoint created earlier by visiting http://localhost:8080/products/es/Samsung.

Test New Endpoint

With these steps, you can ensure that the integration between Spring, Elasticsearch, and Redis is working properly, enabling query-based data retrieval with high responsiveness.

In our project, we successfully integrated Elasticsearch as a search engine with a simple Spring CRUD application. Through steps such as setting up dependencies, customizing the entity model, configuring the repository, adding services for search with queries, and running the entire infrastructure using Docker Compose, this project provides a complete overview of the Elasticsearch implementation.

Elasticsearch offers great benefits in the industrialized world with several key advantages. The search engine is capable of handling and providing fast access to large and complex data. With features such as efficient search processing, deep analytics capabilities, and ease of scalability, Elasticsearch is an invaluable tool in simplifying data search and analysis across industries. Its implementation can positively impact the performance, scalability and accuracy of real-time accessible information, helping companies make smarter data-driven decisions and be responsive to market changes. Therefore, the use of Elasticsearch in the industry is not just an option, but a necessity to improve the efficiency, accuracy, and competitiveness of companies in this digital era.