问题
springboot 版本:spring-boot 2.3.12
今天在开发一个redis 热key服务端的过程中,碰到2个问题:
- jdk8的LocalDateTime,LocalDate,LocalTime类型的字段在序列化,会被序列成["2021","12","22","18","56","40"]这样的数组;
- 服务接受请求LocalDateTime类型的参数时,要求参数为 "2021-12-22T18:56",中间加"T"(ISO-8601) 才能够正常实现反序列化,yyyy-MM-dd HH:mm:ss 格式的字符串反序列化会报异常,异常信息如下:
org.springframework.http.converter.HttpMessageNotReadableException: Invalid JSON input: Cannot deserialize value of type `java.time.LocalDateTime` from String "2021-12-22 18:56:40": Failed to deserialize java.time.LocalDateTime: (java.time.format.DateTimeParseException) Text '2020-05-04 00:00' could not be parsed at index 10; nested exception is com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.time.LocalDateTime` from String "2021-12-22 18:56:40": Failed to deserialize java.time.LocalDateTime: (java.time.format.DateTimeParseException) Text '2021-12-22 18:56:40' could not be parsed at index 10 // 省略部分异常信息 Caused by: java.time.format.DateTimeParseException: Text '2021-12-22 18:56:40' could not be parsed at index 10
系统默认的序列化&反序列化方式给人感觉比较反人类,给所有功能相关都会带来困惑和额外的转化工作量,需要使用一种更符合大家使用习惯的方式解决一下。
方案职责的定位
LocalDateTime序列化&反序列化的使用应该是应用服务的共性问题,发挥作用的层次在springmvc 的HttpMessageConverter层次,个人想法-解决方案应该放在基础框架或脚手架层次(如果所在公司又自己的框架或脚手架),这样所有使用框架及脚手架的应用都会因此受益。
解决过程
定位问题
spring boot 在mvc请求的处理过程中,负责json 格式序列化和反序列化的是Jackson*HttpMessageConverter,具体哪个类定位一下即可,所有的请求response在spring boot里统一在RequestResponseBodyMethodProcessor这个类的handleReturnValue方法中进行的。
@Override public void handleReturnValue(@Nullable Object returnValue, MethodParameter returnType, ModelAndViewContainer mavContainer, NativeWebRequest webRequest) throws IOException, HttpMediaTypeNotAcceptableException, HttpMessageNotWritableException { mavContainer.setRequestHandled(true); ServletServerHttpRequest inputMessage = createInputMessage(webRequest); ServletServerHttpResponse outputMessage = createOutputMessage(webRequest); //这里即是使用各种MessageConverter对返回的对象进行序列化处理的地方 writeWithMessageConverters(returnValue, returnType, inputMessage, outputMessage); }
顺着writeWithMessageConverters方法继续往下debug,最终找到了Jackson的converter -> AbstractJackson2HttpMessageConverter的withInternal方法。
@Override protected void writeInternal(Object object, @Nullable Type type, HttpOutputMessage outputMessage) throws IOException, HttpMessageNotWritableException { MediaType contentType = outputMessage.getHeaders().getContentType(); JsonEncoding encoding = getJsonEncoding(contentType); OutputStream outputStream = StreamUtils.nonClosing(outputMessage.getBody()); JsonGenerator generator = this.objectMapper.getFactory().createGenerator(outputStream, encoding); try { //此处忽略不相关的代码 //此处即是将返回对象序列化的objectWriter ObjectWriter objectWriter = (serializationView != null ? this.objectMapper.writerWithView(serializationView) : this.objectMapper.writer()); if (filters != null) { objectWriter = objectWriter.with(filters); } if (javaType != null && javaType.isContainerType()) { objectWriter = objectWriter.forType(javaType); } SerializationConfig config = objectWriter.getConfig(); if (contentType != null && contentType.isCompatibleWith(MediaType.TEXT_EVENT_STREAM) && config.isEnabled(SerializationFeature.INDENT_OUTPUT)) { objectWriter = objectWriter.with(this.ssePrettyPrinter); } //此处开始进行序列化 objectWriter.writeValue(generator, value); writeSuffix(generator, object); generator.flush(); generator.close(); } catch (InvalidDefinitionException ex) { throw new HttpMessageConversionException("Type definition error: " + ex.getType(), ex); } catch (JsonProcessingException ex) { throw new HttpMessageNotWritableException("Could not write JSON: " + ex.getOriginalMessage(), ex); } }
objectWriter是真正执行序列化的,它是由ObjectMapper来创建的,再来看以下ObjectMapper,
关键的序列化组件由_serializerFactory提供,由内部提供了很多类型对象序列化支撑。
再继续跟踪,最终进入了LocalDateTime类型的序列化类->LocalDateTimeSerializer,通过serialize方法进行序列化,在包里我们还可以看到很多JSR310的日期类型的序列化类。
再来看serialize方法,其中有一个很重要的逻辑->useTimestamp方法,在父类JSR310FormattedSerializerBase中实现,单独摘出来
protected boolean useTimestamp(SerializerProvider provider) { if (_useTimestamp != null) { return _useTimestamp.booleanValue(); } if (_shape != null) { if (_shape == Shape.STRING) { return false; } if (_shape == Shape.NUMBER_INT) { return true; } } //这里让人眼前一亮,意味着可以通过外置扩展的方式来给一个配置好的_formatter return (_formatter == null) && (provider != null) && provider.isEnabled(getTimestampsFeature()); }
这个方法默认返回true,默认_formatter是null,provider.isEnabled也是true,这里我们应该已经找到解题之道了—我们可以试图给-LocalDateTimeSerializer一个定义好的_formatter,或者给定配置让它生成一个formatter.
整理一下调用链路
整理一下思路:LocalDateTimeSerializer->...->DefaultSerializerProvider->Prefetch->ObjectWritter->ObjectMapper
看一下不起眼的Prefetch的关键方法-_serializerProvider
protected DefaultSerializerProvider _serializerProvider() { return _serializerProvider.createInstance(_config, _serializerFactory); }
通过此方法将ObjectWritter中的_serializerFactory来创建一个默认的DefaultSerializerProvider。
好了,回过头来溯源已经清楚了,其实还是ObjectMapper来传入进去的。那么我们就研究在ObjectMapper创建时如何做一下扩展,将自己的扩展解决方案融入到ObjectMapper中就可以了。
解决方案
AbstractJackson2HttpMessageConverter的实现类MappingJackson2HttpMessageConverter,找到这个bean的创建位置
再看一下ObjectMapper的创建位置
如图所示,MappingJackson2HtttMessageConverter的ObjectMapper在JacksonAutoConfiguration中创建的,并且通过Jackson2ObjectMapperBuilder 来创建的,这个类很重要,我们看一下这个build方法。
public <T extends ObjectMapper> T build() { ObjectMapper mapper; if (this.createXmlMapper) { mapper = (this.defaultUseWrapper != null ? new XmlObjectMapperInitializer().create(this.defaultUseWrapper, this.factory) : new XmlObjectMapperInitializer().create(this.factory)); } else { mapper = (this.factory != null ? new ObjectMapper(this.factory) : new ObjectMapper()); } //这里的configure就是配置mapper的,做包括序列在内的各种配置的 configure(mapper); return (T) mapper; }
configure方法:
public void configure(ObjectMapper objectMapper) { Assert.notNull(objectMapper, "ObjectMapper must not be null"); MultiValueMap<Object, Module> modulesToRegister = new LinkedMultiValueMap<>(); if (this.findModulesViaServiceLoader) { ObjectMapper.findModules(this.moduleClassLoader).forEach(module -> registerModule(module, modulesToRegister)); } else if (this.findWellKnownModules) { registerWellKnownModulesIfAvailable(modulesToRegister); } if (this.modules != null) { this.modules.forEach(module -> registerModule(module, modulesToRegister)); } if (this.moduleClasses != null) { for (Class<? extends Module> moduleClass : this.moduleClasses) { registerModule(BeanUtils.instantiateClass(moduleClass), modulesToRegister); } } List<Module> modules = new ArrayList<>(); for (List<Module> nestedModules : modulesToRegister.values()) { modules.addAll(nestedModules); } objectMapper.registerModules(modules); //此处省去不关注的代码 }
这个Module是个抽象类,有这么多实现,JavaTimeModule就是时间相关的模块
看一下JavaTimeModule说明
Class that registers capability of serializing java.time objects with the Jackson core.
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JavaTimeModule());
Note that as of 2.x, if auto-registering modules, this package will register legacy version, JSR310Module, and NOT this module. 3.x will change the default. Legacy version has the same functionality, but slightly different default configuration: see JSR310Module for details.
再看一下JavaTimeModule的核心代码,它的构造方法里加入了这么多的序列化及反序列化类,不过是都是默认的,我们需要使用JavaTimeModule创建一个我们需要的类。
public final class JavaTimeModule extends SimpleModule { private static final long serialVersionUID = 1L; public JavaTimeModule() { super(PackageVersion.VERSION); // First deserializers // // Instant variants: addDeserializer(Instant.class, InstantDeserializer.INSTANT); addDeserializer(OffsetDateTime.class, InstantDeserializer.OFFSET_DATE_TIME); addDeserializer(ZonedDateTime.class, InstantDeserializer.ZONED_DATE_TIME); // // Other deserializers addDeserializer(Duration.class, DurationDeserializer.INSTANCE); addDeserializer(LocalDateTime.class, LocalDateTimeDeserializer.INSTANCE); addDeserializer(LocalDate.class, LocalDateDeserializer.INSTANCE); addDeserializer(LocalTime.class, LocalTimeDeserializer.INSTANCE); addDeserializer(MonthDay.class, MonthDayDeserializer.INSTANCE); addDeserializer(OffsetTime.class, OffsetTimeDeserializer.INSTANCE); addDeserializer(Period.class, JSR310StringParsableDeserializer.PERIOD); addDeserializer(Year.class, YearDeserializer.INSTANCE); addDeserializer(YearMonth.class, YearMonthDeserializer.INSTANCE); addDeserializer(ZoneId.class, JSR310StringParsableDeserializer.ZONE_ID); addDeserializer(ZoneOffset.class, JSR310StringParsableDeserializer.ZONE_OFFSET); // then serializers: addSerializer(Duration.class, DurationSerializer.INSTANCE); addSerializer(Instant.class, InstantSerializer.INSTANCE); addSerializer(LocalDateTime.class, LocalDateTimeSerializer.INSTANCE); addSerializer(LocalDate.class, LocalDateSerializer.INSTANCE); addSerializer(LocalTime.class, LocalTimeSerializer.INSTANCE); addSerializer(MonthDay.class, MonthDaySerializer.INSTANCE); addSerializer(OffsetDateTime.class, OffsetDateTimeSerializer.INSTANCE); addSerializer(OffsetTime.class, OffsetTimeSerializer.INSTANCE); addSerializer(Period.class, new ToStringSerializer(Period.class)); addSerializer(Year.class, YearSerializer.INSTANCE); addSerializer(YearMonth.class, YearMonthSerializer.INSTANCE);
回过头来继续看Jackson2ObjectMapperBuilder的创建,在JacksonAutoConfiguration中创建
@Configuration(proxyBeanMethods = false) @ConditionalOnClass(Jackson2ObjectMapperBuilder.class) static class JacksonObjectMapperBuilderConfiguration { @Bean @Scope("prototype") @ConditionalOnMissingBean Jackson2ObjectMapperBuilder jacksonObjectMapperBuilder(ApplicationContext applicationContext, List<Jackson2ObjectMapperBuilderCustomizer> customizers) { //这里有Jackson2ObjectMapperBuilderCustomizer,意味着我们可以通过自定义它的Cutomizer, //来做一下个性化扩展 Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder(); builder.applicationContext(applicationContext); customize(builder, customizers); return builder; } private void customize(Jackson2ObjectMapperBuilder builder, List<Jackson2ObjectMapperBuilderCustomizer> customizers) { for (Jackson2ObjectMapperBuilderCustomizer customizer : customizers) { customizer.customize(builder); } } }
熟悉spring 设计风格的同学应该知道,这里的Customizer 类似于RestTemplateBuilder中的customizer,是方便我们做扩展用的,这就是spring 牛逼之处,再次深深膜拜一下,了解到这里几乎可以确定,我们扩展一个Jackson2ObjectMapperBuilderCustomizer的实现,就可以达成我们的目的了。
代码实现
给出代码设计
import com.fasterxml.jackson.databind.Module; import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule; import com.fasterxml.jackson.datatype.jsr310.deser.LocalDateDeserializer; import com.fasterxml.jackson.datatype.jsr310.deser.LocalDateTimeDeserializer; import com.fasterxml.jackson.datatype.jsr310.deser.LocalTimeDeserializer; import com.fasterxml.jackson.datatype.jsr310.ser.LocalDateSerializer; import com.fasterxml.jackson.datatype.jsr310.ser.LocalDateTimeSerializer; import com.fasterxml.jackson.datatype.jsr310.ser.LocalTimeSerializer; import org.springframework.beans.factory.BeanFactoryUtils; import org.springframework.beans.factory.ListableBeanFactory; import org.springframework.boot.autoconfigure.jackson.Jackson2ObjectMapperBuilderCustomizer; import org.springframework.boot.autoconfigure.jackson.JacksonProperties; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.core.Ordered; import org.springframework.http.converter.json.Jackson2ObjectMapperBuilder; import org.springframework.util.CollectionUtils; import java.time.LocalDate; import java.time.LocalDateTime; import java.time.LocalTime; import java.time.format.DateTimeFormatter; import java.util.Collection; /** * @author zhaoxinbo * @name: ApplicationWebConfig * @description: web配置类 * @date 2021/12/2120:40 */ @Configuration public class ApplicationWebConfig { private static final String STANDARD_LOCAL_DATETIME_FORMAT = "yyyy-MM-dd HH:mm:ss SSS"; private static final String STANDARD_LOCAL_DATE_FORMAT = "yyyy-MM-dd"; private static final String STANDARD_LOCAL_TIME_FORMAT = "HH:mm:ss SSS"; @Bean public LocalJackson2ObjectMapperBuilderCustomizer localJackson2ObjectMapperBuilderCustomizer(ApplicationContext applicationContext, JacksonProperties jacksonProperties) { return new LocalJackson2ObjectMapperBuilderCustomizer(applicationContext, jacksonProperties); } /** * 创建应用自己的JavaTimeModule * <p> * 1. Jackson默认不会创建spring 容器管理的JavaTimeModule,我们可以创建这样一个实例去覆盖系统默认的; * 2. 在此我们可以自定义JSR310日期的序列化&反序列化对象 * </p> * @return */ @Bean public JavaTimeModule javaTimeModule() { JavaTimeModule module = new JavaTimeModule(); /** serializers */ LocalDateTimeSerializer localDateTimeSerializer = new LocalDateTimeSerializer(DateTimeFormatter.ofPattern(STANDARD_LOCAL_DATETIME_FORMAT)); LocalDateSerializer localDateSerializer = new LocalDateSerializer(DateTimeFormatter.ofPattern(STANDARD_LOCAL_DATE_FORMAT)); LocalTimeSerializer localTimeSerializer = new LocalTimeSerializer(DateTimeFormatter.ofPattern(STANDARD_LOCAL_TIME_FORMAT)); module.addSerializer(LocalDateTime.class, localDateTimeSerializer); module.addSerializer(LocalDate.class, localDateSerializer); module.addSerializer(LocalTime.class, localTimeSerializer); /** deserializers */ LocalDateTimeDeserializer localDateTimeDeserializer = new LocalDateTimeDeserializer(DateTimeFormatter.ofPattern(STANDARD_LOCAL_DATETIME_FORMAT)); LocalDateDeserializer localDateDeserializer = new LocalDateDeserializer(DateTimeFormatter.ofPattern(STANDARD_LOCAL_DATE_FORMAT)); LocalTimeDeserializer localTimeDeserializer = new LocalTimeDeserializer(DateTimeFormatter.ofPattern(STANDARD_LOCAL_TIME_FORMAT)); module.addDeserializer(LocalDateTime.class, localDateTimeDeserializer); module.addDeserializer(LocalDate.class, localDateDeserializer); module.addDeserializer(LocalTime.class, localTimeDeserializer); return module; } /** * <p> * 1. 自定义Jackson2ObjectMapperBuilderCustomizer; * 2. 将自定义创建JavaTimeModule配置在 Jackson2ObjectMapperBuilder 中 * 3. JacksonAutoConfiguration 中在创建ObjectMapper时就会把我们自己的JavaTimeModule初始化为对应的Serializer了 * </p> */ static final class LocalJackson2ObjectMapperBuilderCustomizer implements Jackson2ObjectMapperBuilderCustomizer, Ordered { private final ApplicationContext applicationContext; private final JacksonProperties jacksonProperties; LocalJackson2ObjectMapperBuilderCustomizer(ApplicationContext applicationContext, JacksonProperties jacksonProperties) { this.applicationContext = applicationContext; this.jacksonProperties = jacksonProperties; } @Override public int getOrder() { return Ordered.LOWEST_PRECEDENCE - 100; } @Override public void customize(Jackson2ObjectMapperBuilder builder) { if (this.jacksonProperties.getDefaultPropertyInclusion() != null) { builder.serializationInclusion(this.jacksonProperties.getDefaultPropertyInclusion()); } if (this.jacksonProperties.getTimeZone() != null) { builder.timeZone(this.jacksonProperties.getTimeZone()); } configureModules(builder); } /** * 将自定义创建JavaTimeModule配置在 Jackson2ObjectMapperBuilder * @param builder */ private void configureModules(Jackson2ObjectMapperBuilder builder) { Collection<Module> modules = getBeans(this.applicationContext, Module.class); if(CollectionUtils.isEmpty(modules)) { return; } builder.modulesToInstall(modules.toArray(new Module[0])); } private static <T> Collection<T> getBeans(ListableBeanFactory beanFactory, Class<T> type) { return BeanFactoryUtils.beansOfTypeIncludingAncestors(beanFactory, type).values(); } } }
效果展示
反序列化
发起请求
url:/worker/test/queryByTime
method: post
请求参数:
{ "updateTime": "2021-12-22 17:12:47 599", "appName": "app-worker" }
结果展示
序列化
发起请求
url:127.0.0.1:9000/worker/node/query?appName=app-worker
method: get
结果展示
{ "code": "00000", "data": [ { "id": 507, "appName": "app-worker", "ip": "10.255.22.204", "port": 11111, "createTime": "2021-12-22 17:12:47 000", "updateTime": "2021-12-23 11:10:34 000" }, { "id": 511, "appName": "app-worker", "ip": "172.20.99.148", "port": 11111, "createTime": "2021-12-23 11:03:26 000", "updateTime": "2021-12-23 11:10:35 000" } ] }
其他实现方案
在JSR310属性上使用@Jsonformat注解,在看代码过程中看到这种解决方式,不过所有JSR310属性都要加,不满足要求,这种方式我并未做验证,有空闲时间了可以做一下验证。
@JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss", timezone = "GMT+8")