问题描述
今天发现当天的索引在ES中并没有创建,logstash中不停的报错:
[2021-05-24T05:47:51,904][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({“type”=>“cluster_block_exception”, “reason”=>“blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];”})
[2021-05-24T05:47:51,904][INFO ][logstash.outputs.elasticsearch] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>1}
索引变成了只读:
原因
一旦在存储超过95%的磁盘中的节点上分配了一个或多个分片的任何索引,该索引将被强制进入只读模式
解决办法
经过分析,发现是因为ES所在服务器磁盘空间太低引起,具体解决办法如下:
第一种办法:在kibana开发控制台执行下面语句:
PUT _settings
{
"index": {
"blocks": {
"read_only_allow_delete": null
}
}
}
第二种办法:在es节点服务器上执行:
curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'