基于百度地图SDK和Elasticsearch GEO查询的地理围栏分析系统（2）-查询实现

在上一篇博客中，我们准备好了数据。现在数据已经以我们需要的格式，存放在Elasticsearch中了。

本文讲述如何在Elasticsearch中进行空间GEO查询和聚合查询，以及如何准备ajax接口。

平台的服务端部分使用的springboot+mybatis的基本开发模式。工程结构如下。

可以看到本工程有三个module：

1）moonlight-web是controller和service层的实现；

2）moonlight-dsl封装了ES空间索引查询和聚合查询的方法；

3）moonlight-dao封装了持久化地理围栏的方法。

我们以客户端请求的处理顺序为例进行讲解。

1、controller

在controller层中，我们实现了4个接口，分别是circle、box、polygon、heatmap，也就是圆形圈选，矩形圈选，多边形圈选和热力图。

先看一下代码的具体实现。

@RestController
@RequestMapping("/moonlight")
public class MoonlightController {

    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private MoonlightService moonlightService;

    @RequestMapping(value = "/circle", method = RequestMethod.GET)
    public ResponseEntity<Response> circle(HttpServletRequest request, HttpServletResponse response) {
        String point = request.getParameter("point");
        String radius = request.getParameter("radius");
        try {
            Map<String, Object> result = moonlightService.circle(point, radius);
            logger.info("circle圈选成功, points={}, radius={}, result={}", point, radius, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "circle圈选成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("circle圈选失败, points={}, radius={}, result={}", point, radius, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "circle圈选失败", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/box", method = RequestMethod.GET)
    public ResponseEntity<Response> box(HttpServletRequest request, HttpServletResponse response) {
        String point1 = request.getParameter("point1");
        String point2 = request.getParameter("point2");
        String point3 = request.getParameter("point3");
        String point4 = request.getParameter("point4");
        try {
            Map<String, Object> result = moonlightService.boundingBox(point1, point2, point3, point4);
            logger.info("box圈选成功, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "box圈选成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("box圈选失败, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "box圈选失败", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/polygon", method = RequestMethod.GET)
    public ResponseEntity<Response> polygon(HttpServletRequest request, HttpServletResponse response) {
        List<String> points = new ArrayList<>();
        Enumeration<String> paramNames = request.getParameterNames();
        while (paramNames.hasMoreElements()) {
            String paramName = paramNames.nextElement();
            if (paramName.startsWith("point")) {
                points.add(request.getParameter(paramName));
            }
        }
        try {
            Map<String, Object> result = moonlightService.polygon(points);
            logger.info("polygon圈选成功, points={}, result={}", points, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "polygon圈选成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("polygon圈选失败, points={}, result={}", points, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "polygon圈选失败", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/heatMap", method = RequestMethod.GET)
    public ResponseEntity<Response> heatMap(HttpServletRequest request, HttpServletResponse response) {
        try {
            List<Map<String, Object>> result = moonlightService.heatMap();
            logger.info("heatMap请求成功, result={}", result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "heatMap请求成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("heatMap请求失败, result={}", null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "heatMap请求失败", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }
}

我们以圆形圈选（circle接口）为例，circle接口传入两个参数，一个是point，也就是中心点坐标，一个是radius，也就是半径，它干的事情就是圈选出，point点周围radius长度内的所有订单数据，具体实现是调用了service层的方法，controller得到圈选的数据后就返回了。

下面我们来看一下service层。

2、service

service层是具体业务的实现。我们这里的service仍然比较简单，可以看到只是初始化了esDao的句柄，然后进行es的geo查询。

先看一下具体代码。

@Service
public class MoonlightService {

    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private ESDao esDao;

    public Map<String, Object> circle(String point, String radius) {
        POI center = new POI(point);
        return esDao.circle(center, Double.parseDouble(radius));
    }

    public Map<String, Object> boundingBox(String point1, String point2, String point3, String point4) {
        POI poi1 = new POI(point1);
        POI poi2 = new POI(point2);
        POI poi3 = new POI(point3);
        POI poi4 = new POI(point4);
        POI topLeft = getTopLeft(poi1, poi2, poi3, poi4);
        POI bottomRight = getBottomRight(poi1, poi2, poi3, poi4);
        logger.info("topLeft - lat={}, lng={}, bottomRight - lat={}, lng={}",
                topLeft.getLat(), topLeft.getLng(), bottomRight.getLat(), bottomRight.getLng());
        return esDao.boundingBox(topLeft, bottomRight);
    }

    public Map<String, Object> polygon(List<String> points) {
        List<POI> poiList = new ArrayList<>();
        for (String point : points) {
            POI poi = new POI(point);
            poiList.add(poi);
        }
        return esDao.polygon(poiList);
    }

    public List<Map<String, Object>> heatMap() {
        return esDao.heatMap();
    }

    private POI getTopLeft(POI poi1, POI poi2, POI poi3, POI poi4) {
        POI topLeft = new POI();
        List<Double> latList = new ArrayList<>();
        List<Double> lngList = new ArrayList<>();
        latList.add(poi1.getLat());
        latList.add(poi2.getLat());
        latList.add(poi3.getLat());
        latList.add(poi4.getLat());
        Collections.sort(latList);
        Double minLat = latList.get(0);
        Double maxLat = latList.get(3);

        lngList.add(poi1.getLng());
        lngList.add(poi2.getLng());
        lngList.add(poi3.getLng());
        lngList.add(poi4.getLng());
        Collections.sort(lngList);
        Double minLng = lngList.get(0);
        Double maxLng = lngList.get(3);

        topLeft.setLat(maxLat);
        topLeft.setLng(minLng);
        return topLeft;
    }

    private POI getBottomRight(POI poi1, POI poi2, POI poi3, POI poi4) {
        POI bottomRight = new POI();
        List<Double> latList = new ArrayList<>();
        List<Double> lngList = new ArrayList<>();
        latList.add(poi1.getLat());
        latList.add(poi2.getLat());
        latList.add(poi3.getLat());
        latList.add(poi4.getLat());
        Collections.sort(latList);
        Double minLat = latList.get(0);
        Double maxLat = latList.get(3);

        lngList.add(poi1.getLng());
        lngList.add(poi2.getLng());
        lngList.add(poi3.getLng());
        lngList.add(poi4.getLng());
        Collections.sort(lngList);
        Double minLng = lngList.get(0);
        Double maxLng = lngList.get(3);

        bottomRight.setLat(minLat);
        bottomRight.setLng(maxLng);
        return bottomRight;
    }
}

我们仍然是以圆形圈选为例，可以看到，service代码的逻辑就是，创建出圈选需要的数据接口，然后调用Dao层进行查询就是了。

circle圈选需要的是一个中心点POI类型，和一个Double半径。

box矩形查询需要的是左上坐标点和右下坐标点，里面有两个函数getTopLeft、getBottomRight分别可以求出矩形的左上点和右下点。

polygon多边形查询需要的是一系列点，这些点顺序的连接所绘制出来的图形就是目标多边形。

heatmap热力图什么参数也不要，将返回一定精度的经纬度计数值，后面我们会详述。

之后所有的service都调用了Dao层的es查询逻辑。所以最重要的一部分是esDao的实现，下面我们就来看一看。

3、Dao

Dao层代码是整个项目的核心，包括对Elasticsearch数据进行圈选和聚合两部分，此外就是热力图数据的准备。

先看一下代码。

@Component
public class ESDao {
    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private ESClient esClient;

    public Map<String, Object> circle(POI center, Double radius) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoDistanceRangeQueryBuilder geoDistanceRangeQuery = QueryBuilders.geoDistanceRangeQuery("location")
                .point(center.getLat(),  center.getLng())
                .from("0m")
                .to(String.format("%fm", radius))
                .includeLower(true)
                .includeUpper(true)
                .optimizeBbox("memory")
                .geoDistance(GeoDistance.SLOPPY_ARC);

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public Map<String, Object> boundingBox(POI topLeft,  POI bottomRight) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoBoundingBoxQueryBuilder geoBoundingBoxQuery = QueryBuilders.geoBoundingBoxQuery("location")
                .topLeft(topLeft.getLat(), topLeft.getLng())
                .bottomRight(bottomRight.getLat(), bottomRight.getLng());

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoBoundingBoxQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public Map<String, Object> polygon(List<POI> poiList) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoPolygonQueryBuilder geoPolygonQuery = QueryBuilders.geoPolygonQuery("location");
        for (POI poi : poiList) {
            geoPolygonQuery.addPoint(poi.getLat(), poi.getLng());
        }

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoPolygonQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public List<Map<String, Object>> heatMap() {

        TermQueryBuilder queryBuilder = termQuery("date", "2017-11-24");
        SearchRequestBuilder searchRequestBuilder = esClient.getClient()
                .prepareSearch("moon").setTypes("bj");
        SearchResponse response = searchRequestBuilder
                .setQuery(queryBuilder)
                .setFrom(0).setSize(10000)
                .setExplain(true).execute().actionGet();

        SearchHits hits = response.getHits();
        Map<String, Integer> countMap = new HashMap<>();
        for (SearchHit hit : hits) {
            Map<String, Object> source = hit.getSource();
            Map<String, Double> locationMap = (Map<String, Double>) source.get("location");
            DecimalFormat df = new DecimalFormat("#.000");
            String lat = df.format(locationMap.get("lat"));
            String lon = df.format(locationMap.get("lon"));
            String key = lat+"-"+lon;
            if (countMap.containsKey(key)) {
                countMap.put(key, countMap.get(key) + 1);
            } else {
                countMap.put(key, 1);
            }
        }
        List<Map<String, Object>> result = new ArrayList<>();
        for (Map.Entry<String, Integer> entry : countMap.entrySet()) {
            String lat = entry.getKey().split("-")[0];
            String lon = entry.getKey().split("-")[1];
            Integer count = entry.getValue();
            Map<String, Object> map = new HashMap<>();
            map.put("lat", Double.parseDouble(lat));
            map.put("lng", Double.parseDouble(lon));
            map.put("count", count);
            result.add(map);
        }
        return result;
    }

    private Map<String, Object> agg(SearchRequestBuilder search) {

        Map<String, Object> resultMap = new HashMap<>();

        GroupBy groupBy = new GroupBy(search, "date_group", "date", true);
        groupBy.addSumAgg("pre_total_fee_sum", "pre_total_fee");
        groupBy.addCountAgg("order_id_count", "order_id");
        groupBy.addSumAgg("cancel_count", "type");

        List<String> xAxis = new ArrayList<>();
        List<String> profits = new ArrayList<>();
        List<String> totals = new ArrayList<>();
        List<String> cancelRatios = new ArrayList<>();
        List<Map<String, Object>> details = new ArrayList<>();

        Map<String, Object> groupbyResponse = groupBy.getGroupbyResponse();
        for (Map.Entry<String, Object> entry : groupbyResponse.entrySet()) {
            String date = entry.getKey();
            xAxis.add(date);
            Map<String, String> subAggMap = (Map<String, String>) entry.getValue();
            String profit = subAggMap.get("pre_total_fee_sum");
            profits.add(profit);
            String total = subAggMap.get("order_id_count");
            totals.add(total);
            String cancelRatioDouble = new DecimalFormat("#.0000").format(
                    Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count"))
            );
            String cancelRatio = new DecimalFormat("0.00%").format(
                    Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count"))
            );
            cancelRatios.add(cancelRatioDouble);

            Map<String, Object> tempMap = new HashMap<>();
            tempMap.put("profit", profit);
            tempMap.put("total", total);
            tempMap.put("cancelRatio", cancelRatio);
            tempMap.put("date", date);
            details.add(tempMap);
        }

        resultMap.put("xAxis", xAxis);
        resultMap.put("profit", profits);
        resultMap.put("total", totals);
        resultMap.put("cancelRatio", cancelRatios);
        resultMap.put("detail", details);

        return resultMap;
    }

}

es圈选部分

circle为例，我们构造了一个geoDistanceRangeQuery查询，这个查询到上一篇博客准备好的moon索引，bj type中去将数据圈选出来。

类似的我们有矩形geoBoundingBoxQuery查询，多边形geoPolygonQuery查询，具体构造查询的方式可以参照代码，这个代码还是很简单的，熟悉es的同学很快可以上手并且实现这样的查询，不熟悉的话可以自行百度一下。如果还有其他的查询条件，可以通过QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery)加入，例如我这里在圈选之外加入了一个terms查询，这个查询相当于sql中的where product_id in (3,4) and ...。

es聚合部分

es聚合部分做的事情是，对查询出的订单进行了聚合运算，例如求和和计数，是两个最常见的运算，这部分在这里不详细叙述了，请参见这篇博客。

热力图

这里要额外说明的是，热力图heatmap，和圈选不一样，他是查询了最近一天type=bj分区里的所有数据，按照坐标进行了计数，可以看到的是，计数的时候，我们指定了精度，这里是小数点后三位有效数字

            DecimalFormat df = new DecimalFormat("#.000");
            String lat = df.format(locationMap.get("lat"));
            String lon = df.format(locationMap.get("lon"));
            String key = lat+"-"+lon;

然后将计数结果返回。百度地图SDK会将计数结果绘制成热力图，这个不用我们管，我会在另一篇博客中讲述这个过程。

到这里，整个工程的基本功能就介绍完了。