转 Shield Your Kibana Dashboards

本文涉及的产品
检索分析服务 Elasticsearch 版,2核4GB开发者规格 1个月
简介:

You work with sensitive data in Elasticsearch indices that you do not want everyone to see in their Kibana dashboards. Like a hospital with patient names. You could give each department their own Elasticsearch cluster in order to prevent all departments to see the patient's names, for example.

But wouldn't it be great if there was only one Elasticsearch cluster and every departments could manage their own Kibana dashboards? And still have the security in place to prevent leaking of private data?

With Elasticsearch Shield, you can create a configurable layer of security on top of your Elasticsearch cluster.In this article, we will explore a small example setup with Shield and Kibana.

In this article, we'll use Elasticsearch 1.4.4, Shield 1.0.1 and Kibana 4.0.0. These are at the time of writing the most-recent  versions. Beginner knowledge of Elasticsearch is expected.

Suppose we want to keep our patient’s names private. An index of patients will be available only for one department. An index of cases will be available to all departments.

First, we'll add some test data to the cluster. Create a file hospital.json with the following content:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
"index"  : {  "_index"  "cases" "_type"  "case" "_id"  "101"  } }
"admission"  "2015-01-03" "discharge"  "2015-01-04" "injury"  "broken arm"  }
"index"  : {  "_index"  "cases" "_type"  "case" "_id"  "102"  } }
"admission"  "2015-01-03" "discharge"  "2015-01-06" "injury"  "broken leg"  }
"index"  : {  "_index"  "cases" "_type"  "case" "_id"  "103"  } }
"admission"  "2015-01-06" "discharge"  "2015-01-07" "injury"  "broken nose"  }
"index"  : {  "_index"  "cases" "_type"  "case" "_id"  "104"  } }
"admission"  "2015-01-07" "discharge"  "2015-01-07" "injury"  "bruised arm"  }
"index"  : {  "_index"  "cases" "_type"  "case" "_id"  "105"  } }
"admission"  "2015-01-08" "discharge"  "2015-01-10" "injury"  "broken arm"  }
"index"  : {  "_index"  "patients" "_type"  "patient" "_id"  "101"  } }
"name"  "Adam" "age"  28  }
"index"  : {  "_index"  "patients" "_type"  "patient" "_id"  "102"  } }
"name"  "Bob" "age"  45  }
"index"  : {  "_index"  "patients" "_type"  "patient" "_id"  "103"  } }
"name"  "Carol" "age"  34  }
"index"  : {  "_index"  "patients" "_type"  "patient" "_id"  "104"  } }
"name"  "David" "age"  14  }
"index"  : {  "_index"  "patients" "_type"  "patient" "_id"  "105"  } }
"name"  "Eddie" "age"  72  }

Then bulk index these documents in the cluster:

?
1
$ curl -X POST  'http://localhost:9200/_bulk'  --data-binary @. /hospital .json

Without security, every user can access all documents. Let's install Shield to add security.

Directions on how to install Shield can be found at the Elasticsearch website . We will do this step by step.

Shield is a commercial product, so first we need to install the license manager:

?
1
2
3
4
5
$ elasticsearch /bin/plugin  -i elasticsearch /license/latest
-> Installing elasticsearch /license/latest ...
Trying http: //download .elasticsearch.org /elasticsearch/license/license-latest .zip...
Downloading .....................................DONE
Installed elasticsearch /license/latest  into  /home/patrick/blog/elasticsearch/plugins/license

Now install Shield itself in the same manner:

?
1
2
3
4
5
$ elasticsearch /bin/plugin  -i elasticsearch /shield/latest
-> Installing elasticsearch /shield/latest ...
Trying http: //download .elasticsearch.org /elasticsearch/shield/shield-latest .zip...
Downloading .....................................DONE
Installed elasticsearch /shield/latest  into  /home/patrick/blog/elasticsearch/plugins/shield

You will need to restart the nodes of your cluster to activate the plugins.

In the logs of Elasticsearch you'll see some messages from the new plugins:

?
1
2
3
4
5
6
7
8
9
[2015-02-12 08:18:01,347][INFO ][shield.license ] [node0] enabling license  for  [shield]
[2015-02-12 08:18:01,347][INFO ][license.plugin.core ] [node0] license  for  [shield] - valid
[2015-02-12 08:18:01,355][ERROR][shield.license ] [node0]
#
# Shield license will expire on [Saturday, March 14, 2015]. Cluster health, 
# cluster stats and indices stats operations are blocked on Shield license expiration.
# All data operations (read and write) continue to work. If you have a new license,
# please update it. Otherwise, please reach out to your support contact.
#

Notice that you will get a 30-day trial period to experiment with Shield. Now all our data is protected. See what happens when we try to get a document:

?
1
2
3
4
5
6
$ curl localhost:9200 /cases/case/101 ?pretty= true
{
     "error"  : "AuthenticationException[missing authentication token 
               for  REST request [ /cases/case/1 ]]",
     "status"  : 401
}

We need to add some roles and users to configure the role-based access control of Shield. First we define the roles and their privileges. The definition of these are found in the elasticsearch/config/shield/roles.yml file. Some roles, like admin and user are predefined:

?
1
2
3
4
5
6
7
8
9
10
# All cluster rights
# All operations on all indices
admin:
   cluster: all
   indices:
     '*' : all
# Read-only operations on indices
user:
   indices:
     '*' : read

Let's edit this roles.yml file to describe our needs. We do not want for every user to access all indices, so we'll change user.indices . We'll add the two roles needed for our organization: doctor and nurse. A doctor has more privileges than a nurse. Doctors can access all indices. Nurses can only access the cases index:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Read-only operations on indices
#user:
#  indices:
#     '*' : read
 
# Doctors can access all indices
doctor:
   indices:
     '*' : read
 
# Nurses can only access the cases index
nurse:
   indices:
     'cases' : read

Now that the roles are defined, we can create users that have these roles. Shield provides three realms to store the users: an internal realm, LDAP or Active Directory. For now, we use the internal realm. The realm is configured in elasticsearch/config/elasticsearch.yml . If nothing is explicitly configured, the internal realm is used.

To add users the esusers command line tool can be used. Let's create two users (both with abc123 as password), one for each role:

?
1
2
$ elasticsearch /bin/shield/esusers  useradd  alice -r nurse
$ elasticsearch /bin/shield/esusers  useradd  bob -r doctor

Just to check if the security works:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ curl --user alice:abc123 localhost: 9200 /_count?pretty= true
{
   "count"  5 ,
   "_shards"  : {
     "total"  1 ,
     "successful"  1 ,
     "failed"  0
   }
}
 
$ curl --user bob:abc123 localhost: 9200 /_count?pretty= true
{
   "count"  10 ,
   "_shards"  : {
     "total"  2 ,
     "successful"  2 ,
     "failed"  0
   }
}

Alice can see the 5 cases in our cases index. Bob can see those 5 cases plus the 5 patients in the patients index.

Now it's time to add Kibana in the mix. Instructions to download and install Kibana 4 can be found on the Elasticsearch website.

When we start the Kibana server we notice that Kibana is not allowed access to the Elasticsearch cluster:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ kibana /bin/kibana
{
     "@timestamp" "2015-02-26T08:24:48.958Z" ,
     "level" "fatal" ,
     "message" "AuthenticationException[missing authentication token for REST request [/.kibana/config/_search]]" ,
     "node_env" "production" ,
     "error" : {
         "message" "AuthenticationException[missing authentication token for REST request [/.kibana/config/_search]]" ,
         "name" "Error" ,
         "stack" : "Error: AuthenticationException[missing authentication token  for  REST request [/.kibana /config/_search ]] 
                  at respond ( /home/patrick/kibana/src/node_modules/elasticsearch/src/lib/transport .js:235:15)
                  at checkRespForFailure ( /home/patrick/kibana/src/node_modules/elasticsearch/src/lib/transport .js:203:7)
                  at HttpConnector. ( /home/patrick/kibana/src/node_modules/elasticsearch/src/lib/connectors/http .js:156:7)
                  at IncomingMessage.bound ( /home/patrick/kibana/src/node_modules/elasticsearch/node_modules/lodash-node/modern/internals/baseBind .js:56:17)
                  at IncomingMessage.emit (events.js:117:20)
                  at _stream_readable.js:944:16
                  at process._tickCallback (node.js:442:13)"
     }
}

We need to tell Shield that Kibana is allowed to access our cluster. Extra information of how to let Kibana work with Shield can be found in the Elasticsearch guide .

Shield is shipped with a default configuration for Kibana 4. We find the following role definition in elasticsearch/config/shield/roles.yml.

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# The required role for kibana 4 users
kibana4:
   cluster: cluster:monitor /nodes/info
   indices:
     '*' :
       - indices:admin /mappings/fields/get
       - indices:admin /validate/query
       - indices:data /read/search
       - indices:data /read/msearch
       - indices:admin /get
     '.kibana' :
       - indices:admin /exists
       - indices:admin /mapping/put
       - indices:admin /mappings/fields/get
       - indices:admin /refresh
       - indices:admin /validate/query
       - indices:data /read/get
       - indices:data /read/mget
       - indices:data /read/search
       - indices:data /write/delete
       - indices:data /write/index
       - indices:data /write/update

When the Kibana Server starts it needs to access the .kibana index. So we need to create a user in Shield for Kibana to connect with:

?
1
$ elasticsearch /bin/shield/esusers  useradd  kibana -r kibana4

This account must be configured in Kibana. Modify kibana/conf/kibana.yml :

?
1
2
3
4
5
6
7
8
9
10
11
12
# If your Elasticsearch is  protected  with basic auth,  this  is the user credentials
# used by the Kibana server to perform maintence on the kibana_index at statup. Your Kibana
# users will still need to authenticate with Elasticsearch (which is proxied thorugh
# the Kibana server)
kibana_elasticsearch_username: kibana
kibana_elasticsearch_password: abc123
 
#----------------------------------------------------------------------------
# In newly version of ElasticSearch  2.1 . 0 , you have to use the following way:
elasticsearch.username: kibana
elasticsearch.password: abc123
#----------------------------------------------------------------------------

The Kibana users must have the kibana4 role to be able to work with Kibana. They must be able to store their visualizations and dashboards in the .kibana index:

?
1
2
$ elasticsearch /bin/shield/esusers  roles alice -a kibana4
$ elasticsearch /bin/shield/esusers  roles bob -a kibana4

Since the default kibana4 role has read access on all indices, alice and bob will be granted all access on all indices. Therefore the role permissions must be modified:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# Doctors can access all indices
doctor:
   indices:
     'cases,patients' :
       - indices:admin/mappings/fields/get
       - indices:admin/validate/query
       - indices:data/read/search
       - indices:data/read/msearch
       - indices:admin/get
       
# Nurses can only access the cases index
nurse:
   indices:
     'cases' :
       - indices:admin/mappings/fields/get
       - indices:admin/validate/query
       - indices:data/read/search
       - indices:data/read/msearch
       - indices:admin/get
       
# The required role  for  kibana  4  users
kibana4:
   cluster:
       - cluster:monitor/nodes/info
       - cluster:monitor/health
   indices:
     '.kibana' :
       - indices:admin/exists
       - indices:admin/mapping/put
       - indices:admin/mappings/fields/get
       - indices:admin/refresh
       - indices:admin/validate/query
       - indices:data/read/get
       - indices:data/read/mget
       - indices:data/read/search
       - indices:data/write/delete
       - indices:data/write/index
       - indices:data/write/update
       - indices:admin/create

With this configuration any user with the kibana4 role is able to use Kibana but only sees data that he or she has the proper clearance for.

We can now start the Kibana Server and see that it runs as it should:

?
1
2
3
4
5
6
7
$ kibana /bin/kibana
{
     "@timestamp" "2015-02-26T08:53:18.961Z" ,
     "level" "info" ,
     "message" "Listening on 0.0.0.0:5601" ,
     "node_env" "production"
}

We can open a browser and head to localhost:5601 to open the Kibana web interface. Log in as Alice:

After logging in, Kibana will ask for the index pattern. We'll keep it simple:

Then in the discover tab you can add fields to your view. Notice that Alice only sees cases:

When we log in as Bob our discover tab shows both cases and patients:

To summarize: we added security to Elasticsearch with Shield and configured some users and roles. There's nothing more to it!



原文地址:http://blog.trifork.com/2015/03/05/shield-your-kibana-dashboards/

相关实践学习
使用阿里云Elasticsearch体验信息检索加速
通过创建登录阿里云Elasticsearch集群,使用DataWorks将MySQL数据同步至Elasticsearch,体验多条件检索效果,简单展示数据同步和信息检索加速的过程和操作。
ElasticSearch 入门精讲
ElasticSearch是一个开源的、基于Lucene的、分布式、高扩展、高实时的搜索与数据分析引擎。根据DB-Engines的排名显示,Elasticsearch是最受欢迎的企业搜索引擎,其次是Apache Solr(也是基于Lucene)。 ElasticSearch的实现原理主要分为以下几个步骤: 用户将数据提交到Elastic Search 数据库中 通过分词控制器去将对应的语句分词,将其权重和分词结果一并存入数据 当用户搜索数据时候,再根据权重将结果排名、打分 将返回结果呈现给用户 Elasticsearch可以用于搜索各种文档。它提供可扩展的搜索,具有接近实时的搜索,并支持多租户。
目录
相关文章
|
数据可视化
kibana的安装
kibana的安装
76 0
|
监控 安全 Java
elasticsearch插件三—— Marvel插件安装详解
一、Marvel插件介绍 Marvel插件:在簇中从每个节点汇集数据。这个插件必须每个节点都得安装。
409 0
elasticsearch插件三—— Marvel插件安装详解
|
存储 数据可视化 数据挖掘
安装、使用Kibana
Kibana是一个开源的分析和可视化平台,设计用于和Elasticsearch一起工作。你用Kibana来搜索,查看,并和存储在Elasticsearch索引中的数据进行交互。你可以轻松地执行高级数据分析,并且以各种图标、表格和地图的形式可视化数据。Kibana使得理解大量数据变得很容易。它简单的、基于浏览器的界面使你能够快速创建和共享动态仪表板,实时显示Elasticsearch查询的变化。
830 0
安装、使用Kibana
|
Docker 容器
安装kibana后的坑
安装kibana后的坑
|
存储 数据可视化 大数据
Kibana的安装
Kibana的安装
|
数据可视化 数据挖掘 索引
安装kibana
Kibana是一款开源的数据分析和可视化平台,设计用于和Elasticsearch协作。我们可以使用Kibana对Elasticsearch索引中的数据进行搜索、查看、交互操作。
|
自然语言处理 监控 网络协议
[基础服务-windows] [ELK] ElasticSearch + Kibana + Logstash 以及插件安装和配置
[基础服务-windows] [ELK] ElasticSearch + Kibana + Logstash 以及插件安装和配置
536 0
[基础服务-windows] [ELK] ElasticSearch + Kibana + Logstash 以及插件安装和配置
|
机器学习/深度学习 Ubuntu iOS开发
【Elastic Engineering】Beats:解密 Filebeat 中的 setup 命令
这个步骤非常重要,但是描述的内容并不是很多。为什么需要这个步骤呢?它到底能够做什么呢?
570 0
【Elastic Engineering】Beats:解密 Filebeat 中的 setup 命令
|
搜索推荐 Apache 文件存储
elaticsearch kibana介绍与安装
elaticsearch kibana介绍与安装
164 0
|
数据可视化 安全 Shell
阿里云安装kibana
Kibana 是一个免费且开放的用户界面,能够让您对 Elasticsearch 数据进行可视化,并让您在 Elastic Stack 中进行导航。您可以进行各种操作,从跟踪查询负载,到理解请求如何流经您的整个应用,都能轻松完成。
386 0