Elasticsearch数据迁移到new_index

启动

brew services start elasticsearch

curl 'http://localhost:9200/?pretty'

问题点

之前是一个 _index，多 _type，多 _id，搜索效率不高，现在想把 多 type 变成 多 index，_type默认，_id 保持不变。

试着用 pyes 写过，不过读不到 hits 里的数据。

reindex

官方文档 https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html

创建数据

1
2
3
4
5
6
7
8
9
POST /_bulk
{"index": {"_index" : "index_test", "_type" : "t1"}}
{"title": "11"}
{"index" : {"_index" : "index_test", "_type" : "t2"}}
{"title": "22"}
{"index" : {"_index" : "index_test", "_type" : "t3"}}
{"title": "33"}
{"index" : {"_index" : "index_test", "_type" : "t4"}}
{"title": "44"}

获取 index_test 里的所有数据

GET /index_test/_search

截取部分数据

1
2
3
4
5
6
7
8
9
10
11
12
13
"hits": {
    "total": 4,
    "max_score": 1,
    "hits": [
      {
        "_index": "index_test",
        "_type": "t2",
        "_id": "AV-ZYOD2AeLwdZfNb3-K",
        "_score": 1,
        "_source": {
          "title": "22"
        }
      },

使用 reindex 将 index/type里的数据迁移到 type/newtype里

创建新索引

example: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
PUT my_index 
{
  "mappings": {
    "user": { 
      "_all":       { "enabled": false  }, 
      "properties": { 
        "title":    { "type": "text"  }, 
        "name":     { "type": "text"  }, 
        "age":      { "type": "integer" }  
      }
    },
    "blogpost": { 
      "_all":       { "enabled": false  }, 
      "properties": { 
        "title":    { "type": "text"  }, 
        "body":     { "type": "text"  }, 
        "user_id":  {
          "type":   "keyword" 
        },
        "created":  {
          "type":   "date", 
          "format": "strict_date_optional_time||epoch_millis"
        }
      }
    }
  }
}

注意：根据不同的 type，执行多条命令。dest 的索引/类型应该与刚创建的索引保持一致。

1
2
3
4
5
6
7
8
9
10
11
POST _reindex
{
  "source": {
    "index": "index_test",
    "type": "t2"
  },
  "dest": {
    "index": "t2",
    "type": "default"
  }
}

查看 t2 的数据

GET /t2/_search

结果：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": "t2",
        "_type": "default",
        "_id": "AV-ZYOD2AeLwdZfNb3-K",
        "_score": 1,
        "_source": {
          "title": "22"
        }
      }
    ]
  }
}

index 和 type 已经正确迁移了。

数据备份

安装 nodejs

brew install nodejs

安装 elasticdump

npm install elasticdump -g

备份

elasticdump  --input=http://10.0.0.1:9200/index --output=http://10.0.0.2:9200/index --type=data