Spring data requery

Spring Data Requery
Coupang Catalog Platform & Quality

debop@coupang.com , pierceh89@coupang.com

Agenda
• What is Spring Data Projects

• How does Spring Data works?

• Pros/Cons Spring Data JPA

• Introduction of Spring Data Requery

• How does Spring Data Requery works

• Features

• Limitations

• Future works

What is Spring Data Projects
• Thin Layer for various Data Sources include NoSQL

• For lock in Spring Framework

• Features

• Unified Criteria builder (CrudRepository)

• Flexible query methods

• Native, Example, Property, Custom

• Main developer age is 28 at first release

Spring Data Projects
Spring Data JPA
Spring Data Commons
Hibernate / EclipseLink …
MySQL PostgreSQL ElasticSearch …
Vendor Driver
Vendor
Driver
Spring Data
ElasticSearch
…
Application

Unified Criteria
public interface CrudRepository<T, ID extends Serializable>
extends Repository<T, ID> {
<S extends T> S save(S entity);
Optional<T> findById(ID primaryKey);
Iterable<T> findAll();
long count();
void delete(T entity);
boolean existsById(ID primaryKey);
// … more functionality omitted.
}

Query by Property
interface PersonRepository extends Repository<User, Long> {
List<Person> findByEmailAddressAndLastname(EmailAddress emailAddress, String lastname);
// Enables the distinct flag for the query
List<Person> findDistinctPeopleByLastnameOrFirstname(String lastname, String firstname);
List<Person> findPeopleDistinctByLastnameOrFirstname(String lastname, String firstname);
// Enabling ignoring case for an individual property
List<Person> findByLastnameIgnoreCase(String lastname);
// Enabling ignoring case for all suitable properties
List<Person> findByLastnameAndFirstnameAllIgnoreCase(String lastname, String firstname);
// Enabling static ORDER BY for a query
List<Person> findByLastnameOrderByFirstnameAsc(String lastname);
List<Person> findByLastnameOrderByFirstnameDesc(String lastname);
}

How does Spring Data works?
• Analysis Method & Parameters

• Is it select query

• Define @Query ? Native Query

• Provide `Example` ? Query by Example

• Custom method body? Just Execute

• Otherwise ? PartTree analysis

• Is it modifying ?

• Parameter has `Pageable` or `Sort` ?

• Determine Return types

• Collection, Page, Slice, Single Entity, Tuple ?

Spring Data JPA - Classes for Query

Spring Data JPA - Query Executions

Pros/Cons Spring Data JPA
• Pros

• Easy to use for simple query

• Hide complicated Hibernate features

• Reduce Learning efforts

• Cons

• Hide Hibernate Features

• @DynamicInsert, @DynamicUpdate, @LazyCollections

• Not provide StatelessSession for Performance

• ORM not suitable for high throughput env.

static class DeleteExecution extends JpaQueryExecution {
private final EntityManager em;
public DeleteExecution(EntityManager em) {
this.em = em;
}
@Override
protected Object doExecute(AbstractJpaQuery jpaQuery, Object[] values) {
Query query = jpaQuery.createQuery(values);
List<?> resultList = query.getResultList();
for (Object o : resultList) {
em.remove(o);
}
return jpaQuery.getQueryMethod().isCollectionQuery() ? resultList : resultList.size();
}
}
Spring Data JPA - DeleteExecution
Load all entities to delete for
cascading delete

Requery Overview
• ORM Library for Java, Kotlin, Android

• No Reflection (vs Hibernate proxy)

• Typed Query language (vs Hibernate Criteria)

• Upsert/Partial objects refresh

• Compile time entity/query validation (vs Hibernate)

• Entity is stateless (vs Hibernate stateful)

• Thread 에 제한 받지 않음 (JPA EntityManager)

• Support RxJava, Async Operations, Java 8

Why Requery
• Provide benefit of ORM

• Entity Mapping

• Schema Generation

• Compile time error detecting

• Easy to learn

• Performance

• When bulk job, max 100x than JPA

• REST API - 2~10x throughput

• Support Upsert, Lazy loading …

DB: MySQL 7 (Local), CPU Core: 8, OS: Mac, JVM: Java 8
29
6.5
15.75
6.6
13.1
2.3
JPA REQUERY JPA REQUERY JPA REQUERY
INSERT INSERT UPDATE UPDATE DELETE DELETE
Data Count = 10

2153
499
1095
562
1053
26.2
JPA REQUERY JPA REQUERY JPA REQUERY
INSERT INSERT UPDATE UPDATE DELETE DELETE
Data Count = 1000
DB: MySQL 7 (Local), CPU Core: 8, OS: Mac, JVM: Java 8

Requery Build Process - Java
Define Entity Annotation Processing
buildscript {
repositories {
jcenter()
maven { url "https://plugins.gradle.org/m2/" }
}
dependencies {
// for Java apt
classpath "net.ltgt.gradle:gradle-apt-plugin:0.15"
}
}
// lombok을 gradle 에서 사용하기 위한 plugin
plugins {
id 'io.franzbecker.gradle-lombok' version '1.14'
}
// lombok을 gradle 에서 사용하기 위해 annotation process를 설정해주어야 합니다.
compileOnly "org.projectlombok:lombok"
annotationProcessor "org.projectlombok:lombok"
testAnnotationProcessor "org.projectlombok:lombok"
annotationProcessor "io.requery:requery-processor"
testAnnotationProcessor "io.requery:requery-processor"
EntityDataStore<Object>

Requery Build Process - Kotlin
Define Entity Annotation Processing KotlinEntityDataStore<Any>
// for kotlin entity
kapt "io.requery:requery-processor"
kaptTest "io.requery:requery-processor"

IntelliJ IDEA Settings
IDEA 상에서 테스트 작업 시 자동으로
apt task를 먼저 수행해 준다.

Define Entity - Java
@Getter
@Entity(name = "BasicUser", copyable = true)
@Table(name = "basic_user")
public abstract class AbstractBasicUser extends AuditableLongEntity {
@Key
@Generated
protected Long id;
protected String name;
protected String email;
protected LocalDate birthday;
protected Integer age;
@ForeignKey
@OneToOne
protected AbstractBasicLocation address;
@ManyToMany(mappedBy = "members")
protected Set<AbstractBasicGroup> groups;
@Column(unique = true)
protected UUID uuid;
@Override
public int hashCode() {
return Objects.hash(name, email, birthday);
}
@Transient
@Override
protected @NotNull ToStringBuilder buildStringHelper() {
return super.buildStringHelper()
.add("name", name)
.add("email", email)
.add("birthday", birthday);
}
private static final long serialVersionUID = -2693264826800934057L;
}

Define Entity - Kotlin
@Entity(model = "functional")
interface Person: Persistable {
@get:Key
@get:Generated
val id: Long
@get:Index(value = ["idx_person_name_email"])
var name: String
@get:Index(value = ["idx_person_name_email", "idx_person_email"])
var email: String
var birthday: LocalDate
@get:Column(value = "'empty'")
var description: String?
@get:Nullable
var age: Int?
@get:ForeignKey
@get:OneToOne(mappedBy = "person", cascade = [CascadeAction.DELETE, CascadeAction.SAVE])
var address: Address?
@get:OneToMany(mappedBy = "owner", cascade = [CascadeAction.DELETE, CascadeAction.SAVE])
val phoneNumbers: MutableSet<Phone>
@get:OneToMany
val phoneNumberList: MutableList<Phone>
@get:ManyToMany(mappedBy = "members")
val groups: MutableResult<Group>
@get:ManyToMany(mappedBy = "owners")
val ownedGroups: MutableResult<Group>
@get:ManyToMany(mappedBy = "id")
@get:JunctionTable
val friends: MutableSet<Person>
@get:Lazy
var about: String?
@get:Column(unique = true)
var uuid: UUID
var homepage: URL
var picture: String
}

EntityDataStore<Object>
• findByKey

• select / insert / update / upsert / delete

• where / eq, lte, lt, gt, gte, like, in, not …

• groupBy / having / limit / offset

• support SQL Functions

• count, sum, avg, upper, lower …

• raw query

@Test
fun `insert user`() {
val user = RandomData.randomUser()
withDb(Models.DEFAULT) {
insert(user)
assertThat(user.id).isGreaterThan(0)
val loaded = select(User::class) where (User::id eq user.id) limit 10
assertThat(loaded.get().first()).isEqualTo(user)
}
}
val result = select(Location::class)
.join(User::class).on(User::location eq Location::id)
.where(User::id eq user.id)
.orderBy(Location::city.desc())
.get()
val result = raw(User::class, "SELECT * FROM Users")
val rowCount = update(UserEntity::class)
.set(UserEntity.ABOUT, "nothing")
.set(UserEntity.AGE, 50)
.where(UserEntity.AGE eq 100)
.get()
.value()
val count = insert(PersonEntity::class, PersonEntity.NAME, PersonEntity.DESCRIPTION)
.query(select(GroupEntity.NAME, GroupEntity.DESCRIPTION))
.get()
.first()
.count()

CoroutineEntityDataStore
val store = CoroutineEntityStore(this)
runBlocking {
val users = store.insert(RandomData.randomUsers(10))
users.await().forEach { user ->
assertThat(user.id).isGreaterThan(0)
}
store
.count(UserEntity::class)
.get()
.toDeferred()
.await()
.let {
assertThat(it).isEqualTo(10)
}
}
with(coroutineTemplate) {
val user = randomUser()
// can replace with `withContext { }`
async { insert(user) }.await()
assertThat(user.id).isNotNull()
val group = RandomData.randomGroup()
group.members.add(user)
async { insert(group) }.await()
assertThat(user.groups).hasSize(1)
assertThat(group.members).hasSize(1)
}

spring-data-requery
• RequeryOperations

• Wrap EntityDataStore

• RequeryTransactionManager for TransactionManager

• Support Spring @Transactional

• Better performance than spring-data-jpa

• when exists, paging, not load all entities

spring-data-requery
• Repository built in SQL

• ByPropertyName Auto generation methods

• @Query for Native SQL Query

• Query By Example

• Not Supported

• Association Path (not specified join method)

• Named parameter in @Query (just use `?`)

Spring Data Requery - Repository

Setup spring-data-requery
@Configuration
@EnableTransactionManagement
public class RequeryTestConfiguration extends AbstractRequeryConfiguration {
@Override
@Bean
public EntityModel getEntityModel() {
return Models.DEFAULT;
}
@Override
public TableCreationMode getTableCreationMode() {
return TableCreationMode.CREATE_NOT_EXISTS;
}
@Bean
public DataSource dataSource() {
return new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.build();
}
}

Provided Beans
@Bean
public io.requery.sql.Configuration requeryConfiguration() {
return new ConfigurationBuilder(dataSource, getEntityModel())
// .useDefaultLogging()
.setEntityCache(new EmptyEntityCache())
.setStatementCacheSize(1024)
.setBatchUpdateSize(100)
.addStatementListener(new LogbackListener())
.build();
}
@Bean
public EntityDataStore<Object> entityDataStore() {
log.info("Create EntityDataStore instance.");
return new EntityDataStore<>(requeryConfiguration());
}
@Bean
public RequeryOperations requeryOperations() {
log.info("Create RequeryTemplate instance.");
return new RequeryTemplate(entityDataStore(), requeryMappingContext());
}
EntityCache 설정Tip :
개발 시에는 EmptyEntityCache,
운영 시에는 Cache2kEntityCache 사용

Use @Query in Repository
interface DeclaredQueryRepository extends RequeryRepository<BasicUser, Long> {
@Query("select * from basic_user u where u.email = ?")
BasicUser findByAnnotatedQuery(String email);
@Query("select * from basic_user u where u.email like ?")
List<BasicUser> findAllByEmailMatches(String email);
@Query("select * from basic_user u limit ?")
List<BasicUser> findWithLimits(int limit);
@Query("select * from basic_user u where u.name=? and u.email=? limit 1")
BasicUser findAllBy(String name, String email);
@Query("select u.id, u.name from basic_user u where u.email=?")
List<Tuple> findAllIds(String email);
@Query("select * from basic_user u where u.birthday = ?")
List<BasicUser> findByBirthday(LocalDate birthday);
}

Query By Example
BasicUser user = RandomData.randomUser();
user.setName("example");
requeryTemplate.insert(user);
BasicUser exampleUser = new BasicUser();
exampleUser.setName("EXA");
ExampleMatcher matcher = matching()
.withMatcher("name", startsWith().ignoreCase())
.withIgnoreNullValues();
Example<BasicUser> example = Example.of(exampleUser, matcher);
Return<? extends Result<BasicUser>> query = buildQueryByExample(example);
BasicUser foundUser = query.get().firstOrNull();
assertThat(foundUser).isNotNull().isEqualTo(user);

Query by Property
List<User> findByFirstnameOrLastname(String firstname, String lastname);
List<User> findByLastnameLikeOrderByFirstnameDesc(String lastname);
List<User> findByLastnameNotLike(String lastname);
List<User> findByLastnameNot(String lastname);
List<User> findByManagerLastname(String name);
List<User> findByColleaguesLastname(String lastname);
List<User> findByLastnameNotNull();
@Query("select u.lastname from SD_User u group by u.lastname")
Page<String> findByLastnameGrouped(Pageable pageable);
long countByLastname(String lastname);
int countUsersByFirstname(String firstname);
boolean existsByLastname(String lastname);
Note: Association Path is not supported
Note: Association Path is not supported

Limitations
• Not Supported

• Named Parameter in Native Query

• `@Params`

• Association queries -> Use join directly

• Currently based spring-data-commons 2.0.x

• Based Spring Boot 2.x

Future works
• Support spring-data-commons 1.x

• For Old Spring frameworks

• Support Named parameter

• Support `@Param` in spring data

• Requery for SQL on Hadoop

• Apache Phoenix (HBase)

• Apache Hive, Apache Drill …

Resources
• requery.io

• spring-data-requery in Github

Spring data requery

More Related Content

What's hot

Similar to Spring data requery

More from Sunghyouk Bae

Recently uploaded

Spring data requery