一、基本概念
1、Node 與 Cluster
Elastic 本質上是一個分布式資料庫,允許多台伺服器協同工作,每台伺服器可以運作多個 Elastic 執行個體。
單個 Elastic 執行個體稱為一個節點(node)。一組節點構成一個叢集(cluster)。
2、Index
Elastic 會索引所有字段,經過處理後寫入一個反向索引(Inverted Index)。查找資料的時候,直接查找該索引。
是以,Elastic 資料管理的頂層機關就叫做 Index(索引)。它是單個資料庫的同義詞。每個 Index (即資料庫)的名字必須是小寫。
下面的指令可以檢視目前節點的所有 Index。
$ curl -X GET 'http://localhost:9200/_cat/indices?v'
![](https://img.laitimes.com/img/_0nNw4CM6IyYiwiM6ICdiwiI0gTMx81dsQWZ4lmZf1GLlpXazVmcvwFciV2dsQXYtJ3bm9CX9s2RkBnVHFmb1clWvB3MaVnRtp1XlBXe0xCMy81dvRWYoNHLwEzX5xCMx8FesU2cfdGLwMzX0xiRGZkRGZ0Xy9GbvNGLpZTY1EmMZVDUSFTU4VFRR9Fd4VGdsYTMfVmepNHLrJXYtJXZ0F2dvwVZnFWbp1zczV2YvJHctM3cv1Ce-cmbw5CM3UTM3YTO3kTZ1IGM3UGZyYzXzEDMzgTM1IzLcdDMyIDMy8CXn9Gbi9CXzV2Zh1WavwVbvNmLvR3YxUjLyM3Lc9CX6MHc0RHaiojIsJye.png)
3、Document
Index 裡面單條的記錄稱為 Document(文檔)。許多條 Document 構成了一個 Index。
Document 使用 JSON 格式表示,下面是一個例子。
{
"user": "張三",
"title": "工程師",
"desc": "資料庫管理"}
同一個 Index 裡面的 Document,不要求有相同的結構(scheme),但是最好保持相同,這樣有利于提高搜尋效率。
4、Type
Document 可以分組,比如weather這個 Index 裡面,可以按城市分組(北京和上海),也可以按氣候分組(晴天和雨天)。這種分組就叫做 Type,它是虛拟的邏輯分組,用來過濾 Document。
不同的 Type 應該有相似的結構(schema),舉例來說,id字段不能在這個組是字元串,在另一個組是數值。這是與關系型資料庫的表的一個差別。性質完全不同的資料(比如products和logs)應該存成兩個 Index,而不是一個 Index 裡面的兩個 Type(雖然可以做到)。
下面的指令可以列出每個 Index 所包含的 Type。
$ curl 'localhost:9200/_mapping?pretty=true'
根據規劃,Elastic 6.x 版隻允許每個 Index 包含一個 Type,7.x 版将會徹底移除 Type。
以上部分摘自:http://www.ruanyifeng.com/blog/2017/08/elasticsearch.html
二、通過Http請求排序
1、資料格式
{
"_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAATaBwFklfYTRhdy0wVHJxQUNpcm5sWVBHeHcAAAAAAEvhqhYwNTgtVi1xT1FUNlkxMl9CVldWM1lnAAAAAACXzBgWVlhBRnRfd2xRd09HdlduY2tRNXpmQQ==",
"took": 3,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0},
"hits": {
"total": 9564,
"max_score": 1,
"hits": [
{
"_index": "alert-201712s",
"_type": "HISTORY",
"_id": "000E94E15DA381A680F9C0E0C14F1E7F-1513323398",
"_score": 1,
"_source": {
"duration": 120,
"times": 2,
"status": "resolve",
"level": "warning",
"project": "AAAA"}
},
{
"_index": "alert-201712s",
"_type": "HISTORY",
"_id": "00A70A194DCF6DE937BC97610715DDCE-1513320277",
"_score": 1,
"_source": {
"duration": 120,
"times": 54,
"level": "critical",
"project": "BBBB"}
},
..........
]
}
}
想要先按照project聚合,再按照level聚合,再把聚合後的各個項目、各個level的duration求和(類似與sql中的select sum(duration) ….group by project,level)
2、聚合排序
通過postman請求:
請求方式:Post
url:
ip:9200/index名稱/Type名稱/_search
此處應該是:
localhost:9200/alert-201712s/HISTORY/_search
body參數:
{
"size": 0,
"query": {
"bool": {
"filter": {
"terms": {
"project": ["AAAA", "BBBB"] }
}
}
},
"aggs": {
"projects": {
"terms": {
"field": "project",
"size": 10000},
"aggs": {
"levels": {
"terms": { "field": "level"},
"aggs": { "durations": { "sum": { "field": "duration"} } } }
}
}
}
}
body參數注意aggs的嵌套結構(層級)
查詢結果:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0},
"hits": {
"total": 8768,
"max_score": 0,
"hits": []
},
"aggregations": {
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "AAA",
"doc_count": 2077,
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [ { "key": "serious", "doc_count": 789, "durations": { "value": 18720} }, { "key": "null", "doc_count": 456, "durations": { "value": 23} }, { "key": "warning", "doc_count": 401, "durations": { "value": 234} }, { "key": "critical", "doc_count": 4, "durations": { "value": 78} } ] }
},
{
"key": "BBB",
"doc_count": 1225,
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [ { "key": "serious", "doc_count": 966, "durations": { "value": 56} }, { "key": "null", "doc_count": 258, "durations": { "value": 34} }, { "key": "critical", "doc_count": 1, "durations": { "value": 2343} } ] }
}
}
}
三、java http 請求
1、pom依賴
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>5.6.4</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>5.1.1</version>
</dependency>
2、代碼
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.entity.StringEntity;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.transport.TransportClient;
import java.util.HashMap;
import java.util.Map;
public class Test {
private static String es_url = "localhost:9200";
private TransportClient client;
private IndexRequest source;
//将postman中參數直接複制到idea中自動轉義的
private static String str = "{\n" +
"\t\"size\": 0,\n" +
"\t\"query\": {\n" +
"\t\t\"bool\": {\n" +
"\t\t\t\"filter\": {\n" +
"\t\t\t\t\"terms\": {\n" +
"\t\t\t\t\t\"project\": [\"AA\",\n" +
"\t\t\t\t\t\"BB\"]\n" +
"\t\t\t\t}\n" +
"\t\t\t}\n" +
"\t\t}\n" +
"\t},\n" +
"\t\"aggs\": {\n" +
"\t\t\"projects\": {\n" +
"\t\t\t\"terms\": {\n" +
"\t\t\t\t\"field\": \"project\",\n" +
"\t\t\t\t\"size\": 10000\n" +
"\t\t\t},\n" +
"\t\t\t\"aggs\": {\n" +
"\t\t\t\t\"levels\": {\n" +
"\t\t\t\t\t\"terms\": {\n" +
"\t\t\t\t\t\t\"field\": \"level\",\n" +
"\t\t\t\t\t\t\"size\": 10000\n" +
"\t\t\t\t\t},\n" +
"\t\t\t\t\t\"aggs\": {\n" +
"\t\t\t\t\t\t\"durations\": {\n" +
"\t\t\t\t\t\t\t\"sum\": {\n" +
"\t\t\t\t\t\t\t\t\"field\": \"duration\"\n" +
"\t\t\t\t\t\t\t}\n" +
"\t\t\t\t\t\t}\n" +
"\t\t\t\t\t}\n" +
"\t\t\t\t}\n" +
"\t\t\t}\n" +
"\t\t}\n" +
"\t}\n" +
"}";
public static void main(String[] args) throws Exception {
HttpHost[] hosts = new HttpHost[1];
hosts[0] = HttpHost.create(es_url);
//建立ES請求用戶端
RestClient restClient = RestClient.builder(hosts).build();
String index = "alert-201712s";
String type = "HISTORY";
String endpoint = "/" + index + "/" + type + "/_search";
Map params = new HashMap();
StringEntity queryBody = new StringEntity(str, "UTF-8");
Header header = new BasicHeader("content-type", "application/json");
Response response = restClient.performRequest("GET", endpoint, params, queryBody, header);
//System.out.println(response);
String resultJson = EntityUtils.toString(response.getEntity());
Gson gson = new Gson();
//擷取到傳回的資料
JsonObject resultObj = gson.fromJson(resultJson, JsonObject.class);
}
}
個人微信公衆号: