zoukankan      html  css  js  c++  java
  • Elasticsearch初探(一)

    一、安装(Linux篇)

    现在的开源软件越来越成熟,都打好包了,下载后直接解压就可以测试了。

    1 curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz
    2 tar -zxvf elasticsearch-6.3.1.tar.gz 
    3 cd elasticsearch-6.3.1/bin/
    4 ./elasticsearch

    注意:elasticsearch不允许使用root权限启动,所以使用root下载解压的同学,要chown改变一下文件夹的用户权限。

    看到如下信息就说明启动成功了。

    [2018-07-12T11:07:00,877][INFO ][o.e.n.Node               ] [] initializing ...
    [2018-07-12T11:07:00,922][INFO ][o.e.e.NodeEnvironment    ] [4LdtI3q] using [1] data paths, mounts [[/home (/dev/mapper/cl-home)]], net usable_space [229.9gb], net total_space [240.9gb], types [xfs]
    [2018-07-12T11:07:00,923][INFO ][o.e.e.NodeEnvironment    ] [4LdtI3q] heap size [990.7mb], compressed ordinary object pointers [true]
    [2018-07-12T11:07:00,938][INFO ][o.e.n.Node               ] [4LdtI3q] node name derived from node ID [4LdtI3q-T2ehl0Z6Xv6cOA]; set [node.name] to override
    [2018-07-12T11:07:00,939][INFO ][o.e.n.Node               ] [4LdtI3q] version[6.3.1], pid[21521], build[default/tar/eb782d0/2018-06-29T21:59:26.107521Z], OS[Linux/3.10.0-514.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_112/25.112-b15]
    [2018-07-12T11:07:00,939][INFO ][o.e.n.Node               ] [4LdtI3q] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.af3cuwkJ, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/lings/elastic/elasticsearch-6.3.1, -Des.path.conf=/home/lings/elastic/elasticsearch-6.3.1/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
    [2018-07-12T11:07:02,794][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [aggs-matrix-stats]
    [2018-07-12T11:07:02,794][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [analysis-common]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [ingest-common]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [lang-expression]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [lang-mustache]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [lang-painless]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [mapper-extras]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [parent-join]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [percolator]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [rank-eval]
    [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [reindex]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [repository-url]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [transport-netty4]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [tribe]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-core]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-deprecation]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-graph]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-logstash]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-ml]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-monitoring]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-rollup]
    [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-security]
    [2018-07-12T11:07:02,797][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-sql]
    [2018-07-12T11:07:02,797][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-upgrade]
    [2018-07-12T11:07:02,797][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-watcher]
    [2018-07-12T11:07:02,797][INFO ][o.e.p.PluginsService     ] [4LdtI3q] no plugins loaded
    [2018-07-12T11:07:05,500][INFO ][o.e.x.s.a.s.FileRolesStore] [4LdtI3q] parsed [0] roles from file [/home/lings/elastic/elasticsearch-6.3.1/config/roles.yml]
    [2018-07-12T11:07:05,896][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/21588] [Main.cc@109] controller (64 bit): Version 6.3.1 (Build 4d0b8f0a0ef401) Copyright (c) 2018 Elasticsearch BV
    [2018-07-12T11:07:06,171][DEBUG][o.e.a.ActionModule       ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
    [2018-07-12T11:07:06,425][INFO ][o.e.d.DiscoveryModule    ] [4LdtI3q] using discovery type [zen]
    [2018-07-12T11:07:07,061][INFO ][o.e.n.Node               ] [4LdtI3q] initialized
    [2018-07-12T11:07:07,061][INFO ][o.e.n.Node               ] [4LdtI3q] starting ...
    [2018-07-12T11:07:07,194][INFO ][o.e.t.TransportService   ] [4LdtI3q] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
    [2018-07-12T11:07:07,216][WARN ][o.e.b.BootstrapChecks    ] [4LdtI3q] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
    [2018-07-12T11:07:07,216][WARN ][o.e.b.BootstrapChecks    ] [4LdtI3q] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
    [2018-07-12T11:07:10,259][INFO ][o.e.c.s.MasterService    ] [4LdtI3q] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16658751488, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
    [2018-07-12T11:07:10,264][INFO ][o.e.c.s.ClusterApplierService] [4LdtI3q] new_master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16658751488, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16658751488, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
    [2018-07-12T11:07:10,279][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [4LdtI3q] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
    [2018-07-12T11:07:10,280][INFO ][o.e.n.Node               ] [4LdtI3q] started
    [2018-07-12T11:07:10,622][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [4LdtI3q] Failed to clear cache for realms [[]]
    [2018-07-12T11:07:10,636][INFO ][o.e.l.LicenseService     ] [4LdtI3q] license [d717e767-fe3f-4b36-bc85-cad6dbde3efc] mode [basic] - valid
    [2018-07-12T11:07:10,644][INFO ][o.e.g.GatewayService     ] [4LdtI3q] recovered [2] indices into cluster_state
    [2018-07-12T11:07:10,956][INFO ][o.e.c.r.a.AllocationService] [4LdtI3q] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[customer][2], [customer][3]] ...]).

    二、REST使用示例

    1. 查看集群的状态
    curl -X GET "localhost:9200/_cat/health?v"
    epoch      timestamp cluster       status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
    1531365002 11:10:02  elasticsearch yellow          1         1     10  10    0    0       10             0                  -                 50.0%
    注意这个status的状态是yellow,理想的状态应该是green,原因是没有配置集群。
    
    2. 查看节点的状态
    curl -X GET "localhost:9200/_cat/nodes?v"
    ip        heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
    127.0.0.1           20          57   3    0.07    0.06     0.06 mdi       *      4LdtI3q
    
    3. 查看所有索引(类比mysql的库)的状态
    curl -X GET 'http://localhost:9200/_cat/indices?v'
    health status index    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
    yellow open   bank     BO1IYzoXSpGGyYP3XKo4bA   5   1       1000            0    475.1kb        475.1kb
    yellow open   customer -8lF3GAFT1WI43gR9DiYpw   5   1          5            0     17.7kb         17.7kb
    有两个索引bank和customer,一个1000条数据,一个有5条数据。
    
    4. 查看索引(customer)的状态
    curl -X PUT "localhost:9200/customer?pretty"
    {
      "error" : {
        "root_cause" : [
          {
            "type" : "resource_already_exists_exception",
            "reason" : "index [customer/-8lF3GAFT1WI43gR9DiYpw] already exists",
            "index_uuid" : "-8lF3GAFT1WI43gR9DiYpw",
            "index" : "customer"
          }
        ],
        "type" : "resource_already_exists_exception",
        "reason" : "index [customer/-8lF3GAFT1WI43gR9DiYpw] already exists",
        "index_uuid" : "-8lF3GAFT1WI43gR9DiYpw",
        "index" : "customer"
      },
      "status" : 400
    }
    
    5. 更新(不存在时则增加)索引的document(类比Mysql的行)
    curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
    {
      "name": "John Doe"
    }
    '
    响应如下:
    {
      "_index" : "customer",
      "_type" : "_doc",
      "_id" : "1",
      "_version" : 8, //每更新一次版本记录就+1
      "result" : "updated",
      "_shards" : {
        "total" : 2,
        "successful" : 1,
        "failed" : 0
      },
      "_seq_no" : 7,
      "_primary_term" : 4
    }
    
    6. 增加索引的document(和PUT增加的区别就是不用指定document的id,会随机生成一个)
    curl -X POST "localhost:9200/customer/_doc?pretty" -H 'Content-Type: application/json' -d'
    {
      "name": "Jane Doe"
    }
    '
    响应如下:
    {
      "_index" : "customer",
      "_type" : "_doc",
      "_id" : "Bu1tjWQBZa8_pSJjP5AF", //随机生成的id
      "_version" : 1,
      "result" : "created",
      "_shards" : {
        "total" : 2,
        "successful" : 1,
        "failed" : 0
      },
      "_seq_no" : 8,
      "_primary_term" : 4
    }
    
    7. 查看指定的document
    curl -X GET "localhost:9200/customer/_doc/1?pretty"
    响应结果
    {
      "_index" : "customer",
      "_type" : "_doc",
      "_id" : "1",
      "_version" : 8,
      "found" : true,
      "_source" : {
        "name" : "John Doe"
      }
    }
    
    8. 删除指定的document
    curl -X DELETE "localhost:9200/customer/_doc/B-1ujWQBZa8_pSJjH5CC?pretty"
    响应结果:
    {
      "_index" : "customer",
      "_type" : "_doc",
      "_id" : "B-1ujWQBZa8_pSJjH5CC",
      "_version" : 3,
      "result" : "not_found",
      "_shards" : {
        "total" : 2,
        "successful" : 1,
        "failed" : 0
      },
      "_seq_no" : 7,
      "_primary_term" : 4
    }
    
    9. 批量插入文本数据
    curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@/home/lings/accounts.json"
    /home/lings/accounts.json为json文件的存放路径
    
    
    10. 按照account_number升幂排序
    curl -X GET "localhost:9200/bank/_search?q=*&sort=account_number:asc&pretty"
    完全等效于这个命令
    curl -X GET "localhost:9200/bank/_search" -H 'Content-Type: application/json' -d'
    {
      "query": { "match_all": {} },
      "sort": [
        { "account_number": "asc" }
      ]
    }
    '
    
    响应结果:
    {
      "took" : 3, //查询花费了3ms
      "timed_out" : false, 
      "_shards" : {
        "total" : 5,
        "successful" : 5,
        "skipped" : 0,
        "failed" : 0
      },
      "hits" : {
        "total" : 1000, //一共有1000条数据
        "max_score" : null,
        "hits" : [
          {
            "_index" : "bank",
            "_type" : "_doc",
            "_id" : "0",
            "_score" : null,
            "_source" : {
              "account_number" : 0,
              "balance" : 16623,
              "firstname" : "Bradshaw",
              "lastname" : "Mckenzie",
              "age" : 29,
              "gender" : "F",
              "address" : "244 Columbus Place",
              "employer" : "Euron",
              "email" : "bradshawmckenzie@euron.com",
              "city" : "Hobucken",
              "state" : "CO"
            },
            "sort" : [
              0
            ]
          },
     ......省略中间数据
           {
            "_index" : "bank",
            "_type" : "_doc",
            "_id" : "9",
            "_score" : null,
            "_source" : {
              "account_number" : 9,
              "balance" : 24776,
              "firstname" : "Opal",
              "lastname" : "Meadows",
              "age" : 39,
              "gender" : "M",
              "address" : "963 Neptune Avenue",
              "employer" : "Cedward",
              "email" : "opalmeadows@cedward.com",
              "city" : "Olney",
              "state" : "OH"
            },
            "sort" : [
              9  //一共只返回了10个document
            ]
          }
        ]
      }
    }
    
    11. 查询住址包含Street单词的所有document
    curl -X GET "localhost:9200/bank/_search?pretty" -H 'Content-Type: application/json' -d'
    {
      "query": { "match": {"address":"Street"} },
      "sort": [
        { "account_number": "desc" }
      ]
    }
    '
    响应如下:
    {
      "took" : 9,
      "timed_out" : false,
      "_shards" : {
        "total" : 5,
        "successful" : 5,
        "skipped" : 0,
        "failed" : 0
      },
      "hits" : {
        "total" : 385, //满足条件的385个,总共还是返回10个document
        "max_score" : null,
        "hits" : [
          {
            "_index" : "bank",
            "_type" : "_doc",
            "_id" : "996",
            "_score" : null,
            "_source" : {
              "account_number" : 996,
              "balance" : 17541,
              "firstname" : "Andrews",
              "lastname" : "Herrera",
              "age" : 30,
    ......
    
    
    12. 查询住址包含Street单词的降幂排序后的2个document
    curl -X GET "localhost:9200/bank/_search?pretty" -H 'Content-Type: application/json' -d'
    {
      "query": { "match": {"address":"Street"} },
      "sort": [
        { "account_number": "desc" }
      ],
      "size":2
    }
    '
    响应如下:
    {
      "took" : 2,
      "timed_out" : false,
      "_shards" : {
        "total" : 5,
        "successful" : 5,
        "skipped" : 0,
        "failed" : 0
      },
      "hits" : {
        "total" : 385,
        "max_score" : null,
        "hits" : [
          {
            "_index" : "bank",
            "_type" : "_doc",
            "_id" : "996",
            "_score" : null,
            "_source" : {
              "account_number" : 996,
              "balance" : 17541,
              "firstname" : "Andrews",
              "lastname" : "Herrera",
              "age" : 30,
              "gender" : "F",
              "address" : "570 Vandam Street",
              "employer" : "Klugger",
              "email" : "andrewsherrera@klugger.com",
              "city" : "Whitehaven",
              "state" : "MN"
            },
            "sort" : [
              996
            ]
          },
          {
            "_index" : "bank",
            "_type" : "_doc",
            "_id" : "994",
            "_score" : null,
            "_source" : {
              "account_number" : 994,
              "balance" : 33298,
              "firstname" : "Madge",
              "lastname" : "Holcomb",
              "age" : 31,
              "gender" : "M",
              "address" : "612 Hawthorne Street",
              "employer" : "Escenta",
              "email" : "madgeholcomb@escenta.com",
              "city" : "Alafaya",
              "state" : "OR"
            },
            "sort" : [
              994
            ]
          }
        ]
      }
    }
  • 相关阅读:
    《jQuery源码解析》读书笔记(第一章:总体架构)
    无线开发学习笔记
    无线开发知识
    underscore源码了解
    Git 速查手册
    Jetpack Compose之隐藏Scaffold的BottomNavigation
    一种封装Retrofit的方法,可以自动解析Gson,回避Method return type must not include a type variable or wildcard: retrofit2.Call<T>的问题
    整理之Java容器
    整理之Fragment
    整理之BroadcaseReceiver
  • 原文地址:https://www.cnblogs.com/yoyotl/p/9305163.html
Copyright © 2011-2022 走看看