ELK2之ELK的语法学习

时间:2021-04-27 16:49:47

1.回顾

(1)es是什么?

es是基于Apache Lucene的开源分布式(全文)搜索引擎,提供简单的RESTful API来隐藏Lucene的复杂性。

es除了全文搜索引擎之外,还可以这样描述它:

分布式的实时文件存储,每个字段都被索引并可被搜索
分布式的实时分析搜索引擎
可以扩展到成百上千台服务器,处理PB级结构化或非结构化数据。

(2)

数据组织

- 物理:节点和分片
- 逻辑:索引、类型、文档

(3)

简单操作

GET
PUT
DELETE

2.ES的增删改查(curd)

(1)删除之前的数据&数据准备(创建数据)

DELETE s18

PUT s18/doc/1
{
  "name":"yangyazhou",
  "age": 81,
  "sex": "",
  "tags": "闷骚",
  "b": "19900715"
}

PUT s18/doc/2
{
  "name":"yangtao",
  "age": 18,
  "sex": "",
  "tags":"",
  "b": "19970521"
}

PUT s18/doc/3
{
  "name":"cancan",
  "age": 16,
  "sex":"",
  "tags":"学习认真",
  "b":"19980101"
}

PUT s18/doc/4
{
  "name":"guchenxu",
  "age": 22,
  "sex": "",
  "tags":"幽默",
  "b":"19930302"
}

PUT s18/doc/5
{
  "name":"yangwenyu",
  "age": 23,
  "sex": "",
  "tags":"正人君子",
  "b":"19941201"
}

运行上边的5个操作

(2)查找数据

GET s18/doc/1

GET s18/doc/_search

# 查询字符串 query string
GET s18/doc/_search?q=age:22

(3)更新,只能更新一个,其他不能更新,(不建议使用)

PUT s18/doc/5
{
  "tags":"帅气"
}

GET s18/doc/5

 ELK2之ELK的语法学习

上图是查看到的数据,下面恢复一下数据

(4)只更新指定字段,其他不做修改

#恢复数据
PUT s18/doc/5 { "name":"yangwenyu", "age": 23, "sex": "男", "tags":"正人君子", "b":"19941201" }

# 修改指定字段使用POST POST s18
/doc/5/_update { "doc": { "tags":"帅气" } }
#查看 GET s18
/doc/5

是否可以按照条件删除?

DELETE s18/doc/5

DELETE s18

 不建议下图的删除方式

POST s18/doc/_delete_by_query?q=age:18

 

ELK2之ELK的语法学习

 

# 查询字符串 query string
GET s18/doc/_search?q=age:22

 

 只需要记忆最简单的就可以了

PUT增加  GET查找  POST修改  DELETE删除

 

3.es查询的两种方式

# 查询的两种方式
# 方式一:查询字符串 query string
GET s18/doc/_search?q=age:22

# 方式二:DSL

GET s18/doc/_search
{
  "query": {
    "match": {
      "age": "18"
    }
  }
}

GET s18/doc/_search
{  "query": {  "match": {  "age": 18  }  } }
#内部已经做好了转化

4.复杂查询之es的match和match_all

match

 
 
GET s18/doc/_search { "query": { "match": { "tags": "浪" } } } 
 
 
# 报错,不能加在列表里边
GET s18/doc/_search
{
  "query": {
    "match": {
      "tags": ["", "闷骚"]
    }
  }
}


#通过空格分隔

GET s18/doc/_search
{
  "query": {
    "match": {
      "tags": "浪 闷骚"
    }
  }
}
#通过逗号分隔
GET s18/doc/_search { "query": { "match": { "tags": "浪,闷骚" } } }
#只要符合上边的一个条件就能返回,只是写法不同内部会做一些转换

match_all的用法

#下面的两种方式是等价的
GET s18/doc/_search GET s18/doc/_search { "query": { "match_all": {} } }

5.es的sort排序(通常以数字排序)年龄,薪水,分数等等

desc表示从大到小,降序

asc表示从小到大,升序

注意:不是所有的字段都能排序,选择有意义的排序

# 排序 sort
GET s18
/doc/_search { "query": { "match_all": {} }, "sort": [ { "age": { "order": "desc" } } ] } GET s18/doc/_search { "query": { "match_all": {} }, "sort": [ { "age": { "order": "asc" } } ] }

6.es的分页(结构化查询条件是可插拔的优点)

GET s18/doc/_search
GET s18/doc/_search
{
  "query": {
    "match_all": {}
  },
  "from": 0,
  "size": 2
}

#上边查找的是第1条和第2条数据 GET s18
/doc/_search { "query": { "match_all": {} }, "from": 2, "size": 2 }

#上边查找的是第3条和第4条数据 GET s18
/doc/_search GET s18/doc/_search { "query": { "match_all": {} }, "from": 4, "size": 10 }
#上边查找的是第5条到底15条数据,没有就取到最大值,如果只有1条就只返回1条

分页就是自定制,从哪显示到哪里的意思.

7.es的bool查询should(or) must(and) must_not(not)

#查询yangwenyu或者18岁
GET s18/doc/_search { "query": { "bool": { "should": [ { "match": { "name": "yangwenyu" } }, { "match": { "age": "18" } } ] } } }
#这个查询出的结果排序,也就是打分机制存在于内部算法中
#查询性别是男的并且年龄81
GET s18/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "age": 81
          }
        },
        {
          "match": {
            "sex": ""
          }
        }
      ]
    }
  }
}

 

# 查询性别既不是男的,又不是18岁: must_not
GET s18/doc/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "match": {
            "sex": ""
          }
        },
        {
          "match": {
            "age": 18
          }
        }
      ]
    }
  }
}

 

# 查询年龄大于20岁的男的文档: gt 大于
GET s18/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "sex": ""
          }
        }
      ],
      "filter": {
        "range": {
          "age": {
            "gt": 20
          }
        }
      }
    }
  }
}

 

# gte 大于等于,查询年龄大于等于23的男的
GET s18/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "sex": ""
          }
        }
      ],
      "filter": {
        "range": {
          "age": {
            "gte": 23
          }
        }
      }
    }
  }
}

 

# 小于lt 查询年龄小于20的女的

GET s18/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "sex": ""
          }
        }
      ],
      "filter": {
        "range": {
          "age": {
              "lt": 20
          }
        }
      }
    }
  }
}

 

# 小于等于lte, 查询年龄小于等于23的男的

GET s18/doc/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "sex": ""
          }
        }
      ],
      "filter": {
        "range": {
          "age": {
            "lte": 23
          }
        }
      }
    }
  }
}

 

# filter中尽量用must,避免脏数据
GET s18/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "sex": ""
          }
        }
      ],
      "filter": {
        "range": {
          "age": {
            "lte": 23
          }
        }
      }
    }
  }
}

 

# 查询年龄小于等于23的非男性

GET s18/doc/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "match": {
            "sex": ""
          }
        }
      ],
      "filter": {
        "range": {
          "age": {
            "lte": 23
          }
        }
      }
    }
  }
}

9.es的高亮查询

关键字高亮显示,查询是哪个检索的.

# 高亮查询
# 查询name是cancan的文档
GET s18/doc/_search
{
  "query": {
    "match": {
      "name": "cancan"
    }
  },
  "highlight": {
    "fields": {
      "name": {}
    }
  }
}


GET s18/doc/_search
{
  "query": {
    "match": {
      "name": "cancan"
    }
  },
  "highlight": {
    "pre_tags": "<b style='color:red;font-size:20px;' class='wangdi'>", 
    "post_tags": "</b>", 
    "fields": {
      "name": {}
    }
  }
}

#现在只是json结果,只有放在前端才能显示结果

PUT s18/doc/7
{
  "name":"wangdi",
  "desc": "骚的打漂"
}


GET s18/doc/_search
{
  "query": {
    "match": {
      "desc": "打漂"
    }
  },
  "highlight": {
    "pre_tags": "<b style='color:red;font-size:20px;' class='wangdi'>", 
    "post_tags": "</b>", 
    "fields": {
      "desc": {}
    }
  }
}

#上边代表只是高亮显示"打漂"

#高亮显示就是重要的点

 

10.es的结果字段过滤

# 结果过滤

GET s18/doc/_search
{
  "query": {
    "match": {
      "name": "yangtao"
    }
  },
  "_source": "name"
}

GET s18
/doc/_search { "query": { "match": { "name": "yangtao" } }, "_source": ["name", "age", "sex"] }

我们只需要过滤出,我们需要的字段,减少服务器压力

 

11.es的聚合

# 聚合查询

# sum,查询所有男生的年龄总和
GET s18/doc/_search
{
  "query": {
    "match": {
      "sex": ""
    }
  },
  "aggs": {
    "my_sum": {
      "sum": {
        "field": "age"
      }
    }
  }
}


# 查询年龄最大的男生 max
GET s18/doc/_search
{
  "query": {
    "match": {
      "sex": ""
    }
  },
  "aggs": {
    "my_max": {
      "max": {
        "field": "age"
      }
    }
  }
}


# 查询年龄最小的 min

GET s18/doc/_search
{
  "aggs": {
    "my_min": {
      "min": {
        "field": "age"
      }
    }
  }
}

# 求平均 avg
GET s18/doc/_search
{
  "aggs": {
    "my_avg": {
      "avg": {
        "field": "age"
      }
    }
  }
}


# 分组,根据年龄,10-20,,20-3030-100,每个年龄段有多少人?

GET s18/doc/_search { "query": { "match": { "sex": "" } }, "aggs": { "my_group":{ "range": { "field": "age", "ranges": [ { "from": 10, "to": 20 }, { "from": 20, "to": 30 }, { "from": 30, "to": 100 } ] } } } } # 分组,根据年龄,10-20,,20-3030-100, 对每组年龄求和 GET s18/doc/_search { "query": { "match": { "sex": "" } }, "aggs": { "group":{ "range": { "field": "age", "ranges": [ { "from": 10, "to": 20 }, { "from": 20, "to": 30 }, { "from": 30, "to": 100 } ] }, "aggs": { "my_sum": { "sum": { "field": "age" } } } } } }

先分组,再聚合

12.es的mappings之dynamic

 

 

 

 

homework:

(1)用py脚本制作一键启动es和kibana

(2)倒排索引,把表画出来