After having read this article, I wish to use Spring to stream database query results directly to a JSON response to ensure constant-memory usage (no greedy loading of a List
in memory).
阅读完本文之后,我希望使用Spring将数据库查询结果直接流式传输到JSON响应,以确保内存使用率不变(内存中没有贪婪的List加载)。
Similar to what is done in the article with Hibernate, I assembled a greetingRepository
object which returns a stream of the database contents based on a JdbcTemplate
. In that implementation, I create an iterator over the queried ResultSet
, and I return the stream as follows:
与使用Hibernate的文章中所做的类似,我组装了一个greetingRepository对象,该对象根据JdbcTemplate返回数据库内容的流。在该实现中,我在查询的ResultSet上创建了一个迭代器,并按如下方式返回流:
return StreamSupport.stream(spliterator(), false).onClose(() -> {
log.info("Closing ResultSetIterator stream");
JdbcUtils.closeResultSet(resultSet);
});
i.e. with an onClose()
method guaranteeing that the underlying ResultSet
will be closed if the stream is declared in a try-with-resources
construct:
即使用onClose()方法,如果在try-with-resources构造中声明了流,则保证将关闭基础ResultSet:
try(Stream<Greeting> stream = greetingRepository.stream()) {
// operate on the stream
} // ResultSet underlying the stream will be guaranteed to be closed
But as in the article, I want this stream to be consumed by a custom object mapper (the enhanced MappingJackson2HttpMessageConverter
defined in the article). If we take the try-with-resources
need aside, this is feasible as follows:
但正如在文章中,我希望这个流由自定义对象映射器(文章中定义的增强的MappingJackson2HttpMessageConverter)使用。如果我们将资源需求放在一边,这可行如下:
@RequestMapping(method = GET)
Stream<GreetingResource> stream() {
return greetingRepository.stream().map(GreetingResource::new);
}
However as a colleague commented at the bottom of that article, this does not take care of closing underlying resources.
然而,正如一位同事在该文章底部评论的那样,这并没有关闭基础资源。
In the context of Spring MVC, how can I stream from the database all the way into a JSON response and still guarantee that the ResultSet
will be closed? Could you provide a concrete example solution?
在Spring MVC的上下文中,我如何从数据库一直流式传输到JSON响应中并仍然保证ResultSet将被关闭?你能提供具体的示例解决方案吗?
1 个解决方案
#1
0
You could create a construct to defer the query execution at the serialization time. This construct will start and end the transaction programmaticaly.
您可以创建一个构造以在序列化时延迟查询执行。此构造将以programmaticaly方式开始和结束事务。
public class TransactionalStreamable<T> {
private final PlatformTransactionManager platformTransactionManager;
private final Callable<Stream<T>> callable;
public TransactionalStreamable(PlatformTransactionManager platformTransactionManager, Callable<Stream<T>> callable) {
this.platformTransactionManager = platformTransactionManager;
this.callable = callable;
}
public Stream stream() {
TransactionTemplate txTemplate = new TransactionTemplate(platformTransactionManager);
txTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
txTemplate.setReadOnly(true);
TransactionStatus transaction = platformTransactionManager.getTransaction(txTemplate);
try {
return callable.call().onClose(() -> {
platformTransactionManager.commit(transaction);
});
} catch (Exception e) {
platformTransactionManager.rollback(transaction);
throw new RuntimeException(e);
}
}
public void forEach(Consumer<T> c) {
try (Stream<T> s = stream()){
s.forEach(c);
}
}
}
Using a dedicated json serializer:
使用专用的json序列化器:
JsonSerializer<?> transactionalStreamableSer = new StdSerializer<TransactionalStreamable<?>>(TransactionalStreamable.class, true) {
@Override
public void serialize(TransactionalStreamable<?> streamable, JsonGenerator jgen, SerializerProvider provider) throws IOException {
jgen.writeStartArray();
streamable.forEach((CheckedConsumer) e -> {
provider.findValueSerializer(e.getClass(), null).serialize(e, jgen, provider);
});
jgen.writeEndArray();
}
};
Which could be used like this:
可以这样使用:
@RequestMapping(method = GET)
TransactionalStreamable<GreetingResource> stream() {
return new TransactionalStreamable(platformTransactionManager , () -> greetingRepository.stream().map(GreetingResource::new));
}
All the work will be done when jackson will serialize the object. It may be or not an issue regarding the error handling (eg. using controller advice).
当杰克逊将序列化对象时,所有工作都将完成。它可能是或者不是关于错误处理的问题(例如,使用控制器建议)。
#1
0
You could create a construct to defer the query execution at the serialization time. This construct will start and end the transaction programmaticaly.
您可以创建一个构造以在序列化时延迟查询执行。此构造将以programmaticaly方式开始和结束事务。
public class TransactionalStreamable<T> {
private final PlatformTransactionManager platformTransactionManager;
private final Callable<Stream<T>> callable;
public TransactionalStreamable(PlatformTransactionManager platformTransactionManager, Callable<Stream<T>> callable) {
this.platformTransactionManager = platformTransactionManager;
this.callable = callable;
}
public Stream stream() {
TransactionTemplate txTemplate = new TransactionTemplate(platformTransactionManager);
txTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
txTemplate.setReadOnly(true);
TransactionStatus transaction = platformTransactionManager.getTransaction(txTemplate);
try {
return callable.call().onClose(() -> {
platformTransactionManager.commit(transaction);
});
} catch (Exception e) {
platformTransactionManager.rollback(transaction);
throw new RuntimeException(e);
}
}
public void forEach(Consumer<T> c) {
try (Stream<T> s = stream()){
s.forEach(c);
}
}
}
Using a dedicated json serializer:
使用专用的json序列化器:
JsonSerializer<?> transactionalStreamableSer = new StdSerializer<TransactionalStreamable<?>>(TransactionalStreamable.class, true) {
@Override
public void serialize(TransactionalStreamable<?> streamable, JsonGenerator jgen, SerializerProvider provider) throws IOException {
jgen.writeStartArray();
streamable.forEach((CheckedConsumer) e -> {
provider.findValueSerializer(e.getClass(), null).serialize(e, jgen, provider);
});
jgen.writeEndArray();
}
};
Which could be used like this:
可以这样使用:
@RequestMapping(method = GET)
TransactionalStreamable<GreetingResource> stream() {
return new TransactionalStreamable(platformTransactionManager , () -> greetingRepository.stream().map(GreetingResource::new));
}
All the work will be done when jackson will serialize the object. It may be or not an issue regarding the error handling (eg. using controller advice).
当杰克逊将序列化对象时,所有工作都将完成。它可能是或者不是关于错误处理的问题(例如,使用控制器建议)。