问题详情

　　查看/var/log/cloudera-scm-server.log的启动日志

问题来源

　　我在用cloudermanager安装好之后，然后，在对如下。

　　配置kerberos后，在web界面重启服务ok。
但有一个警告：Cloudera recommends at least Level 1 TLS when Kerberos is enabled for CDH clusters.

　　设置好上面之后，我再

　 vim /etc/cloudera-scm-agent/config.ini配置文件

　　然后，就再重启service cloudera-scm-server restart 服务，但一直起不来，报错如下：

2016-05-24 15:31:54,843 INFO MainThread:org.mortbay.log: jetty-6.1.26.cloudera.4
2016-05-24 15:31:54,888 WARN MainThread:org.mortbay.log: failed SslSelectChannelConnector@0.0.0.0:7182: java.io.FileNotFoundException: /var/lib/cloudera-scm-server/.keystore (没有那个文件或目录)
2016-05-24 15:31:54,888 WARN MainThread:org.mortbay.log: failed Server@17a7ad7b: java.io.FileNotFoundException: /var/lib/cloudera-scm-server/.keystore (没有那个文件或目录)
2016-05-24 15:31:54,889 ERROR MainThread:com.cloudera.server.cmf.Main: Failed to start Agent listener.
2016-05-24 15:31:54,889 ERROR MainThread:com.cloudera.server.cmf.Main: Server failed.
org.apache.avro.AvroRuntimeException: java.io.FileNotFoundException: /var/lib/cloudera-scm-server/.keystore (No such file or directory)
        at com.cloudera.server.common.HttpConnectorServer.start(HttpConnectorServer.java:89)
        at com.cloudera.server.cmf.Main.startAgentServer(Main.java:571)
        at com.cloudera.server.cmf.Main.startAvro(Main.java:483)
        at com.cloudera.server.cmf.Main.run(Main.java:620)
        at com.cloudera.server.cmf.Main.main(Main.java:217)
Caused by: java.io.FileNotFoundException: /var/lib/cloudera-scm-server/.keystore (No such file  or  directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.(FileInputStream.java:146)
        at org.mortbay.resource.FileResource.getInputStream(FileResource.java:275)
        at org.mortbay.jetty.security.SslSelectChannelConnector.createSSLContext(SslSelectChannelConnector.java:639)
        at org.mortbay.jetty.security.SslSelectChannelConnector.doStart(SslSelectChannelConnector.java:613)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.Server.doStart(Server.java:235)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at com.cloudera.server.common.HttpConnectorServer.start(HttpConnectorServer.java:87)
        ... 4 more

　　但是，我的确，进入/var/lib/cloudera-scm-server目录里去看了。没有 /var/lib/cloudera-scm-server/.keystore 这个文件的确不存在的，要如何解决这个问题呢？

　　原理

　　了解Hadoop Kerberos安全机制
http://www.aboutyun.com/forum.php?mod=viewthread&tid=9943

如何产生，大家可以参考官网：
http://www.cloudera.com/documentation/enterprise/5-2-x/topics/cm_sg_create_key_trust.html

　　下面的解决办法，官网给的太正式了，网上参考资料甚少。

　　倒是，我总结了如下的一个解决办法，大家可以先去尝试下。（我是按照这个方法解决了的）

　　方法1

　　tls安全协议是用的压缩算法它会记录所以可能需要把元数据库全清了重新装了！！！

　　注意：是只删除数据库（比如我这里用的是mysql）的元数据库（即ambari），而不是删除数据库（mysql）啊！！！一定要注意了。

　　方法2（推荐）

　　All Cloudera Manager Server Configuration Properties are inside a embed postgreSQL database. You need change the value true to false in "agent_tls" column -> "configs" table -> "scm" database. Restart Cluster.

http://www.cloudera.com/documentation/enterprise/5-5-x/topics/cm_ig_embed_pstgrs.html

http://www.cloudera.com/documentation/manager/5-1-x/Cloudera-Manager-Configuration-Properties/cm5con...

　　参考来源于

https://community.cloudera.com/t5/Cloudera-Manager-Installation/Cannot-log-in-to-Cloudera-Manager-no-TLS-keystore/td-p/42312

　　国内有人翻译做过来

How to change the Cloudera Manager Properties in postgres

bigdata@nssa-sensor1:~$ sudo mysql -uroot -proot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 329
Server version: 5.5.35-1ubuntu1 (Ubuntu)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| ambari             |
| amon               |
| hive               |
| hue                |
| mysql              |
| oozie              |
| performance_schema |
| scm                |
+--------------------+
9 rows in set (0.00 sec)

mysql> use scm;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+--------------------------------+
| Tables_in_scm                  |
+--------------------------------+
| AUDITS                         |
| CLIENT_CONFIGS                 |
| CLIENT_CONFIGS_TO_HOSTS        |
| CLUSTERS                       |
| CLUSTERS_AUD                   |
| CLUSTER_ACTIVATED_RELEASES     |
| CLUSTER_ACTIVATED_RELEASES_AUD |
| CLUSTER_MANAGED_RELEASES       |
| CLUSTER_UNDISTRIBUTED_RELEASES |
| CM_PEERS                       |
| CM_VERSION                     |
| COMMANDS                       |
| COMMAND_SCHEDULES              |
| CONFIGS                        |
| CONFIGS_AUD                    |
| CONFIG_CONTAINERS              |
| CREDENTIALS                    |
| DIAGNOSTICS_EVENTS             |
| EXTERNAL_ACCOUNTS              |
| EXTERNAL_ACCOUNTS_AUD          |
| GLOBAL_SETTINGS                |
| HOSTS                          |
| HOSTS_AUD                      |
| HOST_TEMPLATES                 |
| HOST_TEMPLATE_TO_ROLE_CONF_GRP |
| METRICS                        |
| PARCELS                        |
| PARCEL_COMPONENTS              |
| PROCESSES                      |
| PROCESS_ACTIVE_RELEASES        |
| RELEASES                       |
| RELEASES_AUD                   |
| REVISIONS                      |
| ROLES                          |
| ROLES_AUD                      |
| ROLE_CONFIG_GROUPS             |
| ROLE_CONFIG_GROUPS_AUD         |
| ROLE_STALENESS_STATUS          |
| SCHEMA_VERSION                 |
| SERVICES                       |
| SERVICES_AUD                   |
| SNAPSHOT_POLICIES              |
| USERS                          |
| USER_ROLES                     |
| USER_SETTINGS                  |
+--------------------------------+
45 rows in set (0.00 sec)

mysql>

　　官网（太复杂）

　　方法一：解决办法

Creating Java Keystores and Truststores（创建Java密钥库和信任库）

Typically, a keystore is used in one of two distinct ways:

The keystore contains private keys and certificates used by SSL servers to authenticate themselves to SSL clients. By convention, such files are referred to as keystores.
When used as a truststore, the file contains certificates of trusted SSL servers, or of Certificate Authorities trusted to identify servers. There are no private keys in the truststore.

翻译就是：

　　密钥库包含SSL服务器使用的私钥和证书，以便向SSL客户端进行身份验证。按照惯例，这样的文件被称为密钥库。
　　当用作信任库时，该文件包含可信SSL服务器或可信任标识服务器的证书颁发机构的证书。信托商店中没有私钥。

　　Note: The foregoing assumes that certificate-based authentication is being used in one direction only—that is, SSL servers are using certificates to authenticate themselves to clients. It is also possible for clients to authenticate themselves to servers using certificates. (This is known as mutual authentication.) Throughout this document, we assume that client certificates are not in use.

　　翻译就是：

　　注意：前面假设仅在一个方向上使用基于证书的身份验证，即SSL服务器正在使用证书向客户端进行身份验证。客户端也可以使用证书对服务器进行身份验证。（这被称为相互认证。）在本文档中，我们假设客户端证书不被使用。

　　While all SSL clients must have access to a truststore, it is not always necessary to create and deploy truststores across a cluster. The standard JDK distribution includes a default truststore which is pre-provisioned with the root certificates of a number of well-known Certificate Authorities. If you do not provide a custom truststore, the Hadoop daemons load this default truststore. Therefore, if you are using certificates issued by a CA in the default truststore, you do not need to provide custom truststores. However, you must consider the following before you decide to use the default truststore:

　　翻译就是：

　　虽然所有SSL客户端都必须具有对信任库的访问权限，但并不总是需要在群集中创建和部署信托库。标准JDK分发包括一个默认的信任商店，该默认信任商店预先配置了许多知名证书颁发机构的根证书。如果您不提供自定义信任库，Hadoop守护进程将加载该默认信任库。因此，如果您使用默认信任库中的CA颁发的证书，则不需要提供自定义的信任库。但是，在决定使用默认信任库之前，您必须考虑以下事项：

If you choose to use the default truststore, it is your responsibility to maintain it. You may need to remove the certificates of CAs you do not deem trustworthy, or add or update the certificates of CAs you trust. Use the keytool utility to perform these actions.

　　翻译就是：

　　如果您选择使用默认信任库，则有责任维护它。您可能需要删除不认为可靠的CA证书，或者添加或更新您信任的CA证书。使用keytool实用程序来执行这些操作。

Continue reading:

　　然后，我们继续阅读下去。

Security Considerations for Keystores and Truststores

　　（密钥库和信任域的安全注意事项）

　　Note: While the strategy for certificate deployment you select will ultimately depend upon the security policies you wish to implement, the following guidelines may prove useful.

　　翻译就是：

　　　　注意：虽然您选择的证书部署策略最终将取决于您希望实施的安全策略，但以下准则可能有用。

Because keystores contain private keys, while truststores do not, the security requirements for keystores are more stringent. In particular:

Hadoop SSL requires that truststores and the truststore password be stored, in plaintext, in a configuration file that is readable by all.
Keystore and key passwords are stored, in plaintext, in a file that is readable only by members of the appropriate group.

These considerations should inform your choice of which keys and certificates to store in the keystores and truststores you will deploy across your cluster.

Keystores should contain a minimal set of keys and certificates. A reasonable strategy would be to create a unique keystore for each host, which would contain only the keys and certificates needed by the Hadoop SSL services running on the host. In most cases, the keystore would contain a single key/certificate entry.
On the other hand, because truststores do not contain sensitive information, it is reasonable to create a single truststore for an entire cluster. On a production cluster, such a truststore would often contain a single CA certificate (or certificate chain), since you would typically choose to have all certificates issued by a single CA.
Important: Do not use the same password for truststores and keystores/keys.Since truststore passwords are stored in the clear in files readable by all, doing so would compromise the security of the private keys in the keystore.

　　翻译就是：

由于密钥库包含私钥，而信任库不存在，密钥库的安全性要求更为严格。尤其是：

　　Hadoop SSL要求信任库和信任密码以明文方式存储在所有可读的配置文件中。
　　密钥库和密钥密码以明文形式存储在只能由适当组的成员读取的文件中。
　　这些注意事项应该通知您选择要在集群中部署的密钥库和信任库中存储哪些密钥和证书。

　　密钥库应包含一组最小的密钥和证书。合理的策略是为每个主机创建唯一的密钥库，其中只包含主机上运行的Hadoop SSL服务所需的密钥和证书。在大多数情况下，密钥库将包含单个密钥/证书条目。
　　另一方面，由于信任库不包含敏感信息，因此为整个集群创建单个信任库是合理的。在生产集群上，这样的信任商店通常会包含单个CA证书（或证书链），因为您通常会选择由单个CA颁发所有证书。
　　重要信息：请勿对信任库和密钥库/密钥使用相同的密码。
　　由于信任密码存储在所有可读的文件中，这样做会危及密钥库中私钥的安全性。

Creating Keystores（床架密钥库）

　　Once you have settled on a storage plan for your keys and certificates, you can use keytool to create or update the necessary keystores and truststores. To create a new keystore with a certificate see Creating Certificates.

　　In many cases, you will already have created the set of keystores that you need. If you have followed the approach of creating a separate keystore for each private key and certificate, and wish to maintain this arrangement when deploying the keystores, no additional steps are required to prepare the keystores for deployment. If you wish to reorganize your keys and certificates into a different set of keystores, you can use keytool -importkeystore to transfer entries from one keystore to another.

　　翻译就是：

　　创建密钥库
　　一旦确定了密钥和证书的存储计划，您可以使用keytool来创建或更新必要的密钥库和信任库。要使用证书创建新密钥库，请参阅 Creating Certificates。

　　在许多情况下，您将已经创建了所需的一组密钥库。如果您遵循为每个私钥和证书创建单独密钥库的方法，并希望在部署密钥库时保持此安排，则无需准备其他部署步骤即可部署密钥库。如果您希望将密钥和证书重组为不同的密钥库，则可以使用keytool -importkeystore将条目从一个密钥库传输到另一个密钥库。

　　然后，接着阅读

Creating Certificates（创建证书）

一、Creating Certificates（创建证书）

The following sections will walk you through obtaining certificates from commercial Certificate Authorities and creating self-signed test certificates.

　　翻译就是：

　　以下部分将引导您从商业证书颁发机构获取证书并创建自签名测试证书。

　　（1）使用Keytool
　　（2）使用OpenSSL
　　（3）从商业CA获得生产证书
　　（4）创建自签名测试证书

Using Keytool（使用keytool）

　　Keytool is a utility for creating and managing certificates and cryptographic keys, and is part of the standard JDK distribution. The keytool executable usually resides in $JAVA_HOME/bin.

　　Keytool stores certificates and keys in a file known as a keystore . While several different keystore types are supported, by default keytool uses the Java KeyStore (JKS) format.

　　Java-based services such as HDFS, MapReduce, and YARN use the JKS format by default. For this reason it is particularly convenient to use keytool for managing keys and certificates for these services. In the following topics, we assume you are using keytool.

　　For additional information on keytool, refer the keytool documentation.

　　翻译就是：

　　Keytool是用于创建和管理证书和加密密钥的实用程序，是标准JDK发行版的一部分。 keytool可执行文件通常驻留在$ JAVA_HOME / bin中。

　　Keytool将证书和密钥存储在称为密钥库的文件中。虽然支持几种不同的密钥库类型，但默认情况下，keytool使用Java KeyStore（JKS）格式。

　　默认情况下，基于Java的服务（如HDFS，MapReduce和YARN）使用JKS格式。因此，使用keytool来管理这些服务的密钥和证书是特别方便的。在以下主题中，我们假设您正在使用keytool。

　　有关keytool的其他信息，请参阅keytool文档。

Using OpenSSL（使用OpenSSL）

　　Python-based services such as Hue expect certificates and keys to be stored in PEM format. When managing certificates and keys for such services, you may find it convenient to use the openssl tool.

　　Refer the openssl documentation for more information.

　　翻译就是：

　　基于Python的服务，如Hue，期望证书和密钥以PEM格式存储。在管理这些服务的证书和密钥时，您可能会发现使用openssl工具很方便。

　　有关更多信息，请参阅openssl文档。

Obtaining a Production Certificate from a Commercial CA（从商业CA获得生产证书）

　　Once you have decided on a certificate-provisioning strategy, and have determined which hosts require certificates, you will typically purchase the necessary certificates from a commercial Certificate Authority (CA). The procedure for applying for a certificate varies from one CA to another, but typically involves providing some form of proof that you are the legitimate owner of the domain name for which you are requesting a certificate, generating a key pair, and submitting a Certificate Signing Request (CSR) to the CA.

　　翻译就是：

　　一旦您决定了证书配置策略，并确定哪些主机需要证书，您通常将从商业认证机构（CA）购买必要的证书。申请证书的过程从一个CA到另一个CA不同，但通常涉及提供某种形式的证明，证明您是要求证书的域名的合法所有者，生成密钥对并提交证书签名请求（CSR）到CA。

　　As noted above, you may find it convenient to use the Java keytool utility to generate your key pair and CSR, and to manage your certificates. The CA you choose will provide instructions for obtaining and installing a certificate; typically, there will be separate sets of instructions for different web and application servers. The instructions for Java-based servers (Tomcat, for example), will usually describe the following process comprising three keytool commands to obtain a certificate:

　　翻译就是：

　　如上所述，您可能会发现使用Java keytool实用程序可以方便地生成密钥对和CSR，并管理证书。您选择的CA将提供获取和安装证书的说明; 通常，对于不同的web和应用服务器将存在单独的指令集。基于Java的服务器（例如，Tomcat）的说明通常将描述以下过程，包括三个keytool命令来获取证书：

keytool -genkeypair to generate a public/private key pair and create the keystore.
keytool -certreq to create the CSR.
keytool -importcert to import the signed certificate into the keystore.

　　翻译就是：

　　　　keytool -genkeypair生成公钥/私钥对并创建密钥库。
　　　　keytool -certreq来创建CSR。
　　　　keytool -importcert将签名的证书导入密钥库。

　　For example, to generate a public/private key pair for the domain name node1.example.com, you would use a command similar to the one shown below:

　　翻译就是：　　

　　例如，要为域名node1.example.com生成公钥/私钥对，您将使用类似于以下所示的命令：

$ keytool -genkeypair -keystore node1.keystore -alias node1 \
-dname "CN=node1.example.com,O=Hadoop" -keyalg RSA \
-keysize 2048 -storepass changeme -keypass changeme

　　This command generates a pair of 2048-bit keys using the RSA key algorithm, one of several available. The keys are stored in a keystore file called node1.keystore, in a keystore entry identified by by the alias node1. The keystore password (which protects the keystore as a whole) and the key password (which protects the private key stored in the node1 entry) are set using the -storepass and -keypass options (respectively). -keypassmust be set to the same password value as -storepass for Cloudera Manager to access the keystore.

　　翻译就是：

　　该命令使用RSA密钥算法生成一对2048位密钥，其中有几种可用。密钥存储在名为node1.keystore的密钥库文件中，位于由别名node1标识的密钥库条目中。分别使用-storepass和-keypass选项设置密钥库密码（保护密钥库作为整体）和密钥密码（保护存储在node1条目中的私钥）。必须将-keypass设置为与Cloudera Manager的-storepass相同的密码值才能访问密钥库。

　　To create a CSR, you would use a command similar to the following:

　　翻译就是:　　

　　要创建CSR，您将使用类似于以下命令：

$ keytool -certreq -keystore node1.keystore -alias node1 \
-storepass changeme -keypass changeme -file node1.csr

　　This command generates the CSR, and stores it in a file called node1.csr. Once you've submitted your CSR to the CA, and received the CA's reply (containing the signed certificate), you will use the following keytool -importcert command to import the reply into your keystore:

　　翻译就是：　　

　　此命令生成CSR，并将其存储在名为node1.csr的文件中。一旦您将CSR提交给CA，并收到CA的回复（包含签名的证书），您将使用以下keytool -importcert命令将回复导入您的密钥库：

$ keytool -importcert -keystore node1.keystore -alias node1 \
-storepass changeme -keypass changeme -trustcacerts -file node1.crt

　　Here we assume that the CA's reply is stored in the file node1.crt.

　　Important: This section describes a generic procedure using keytool to obtain a certificate from a commercial Certificate Authority. This procedure will differ from one CA to another and Cloudera recommends you consult your CA's documentation for more specifics.

　　翻译就是：　　

　　这里我们假设CA的回复存储在文件node1.crt中。

　　要点：本节介绍使用keytool从商业证书颁发机构获取证书的一般过程。此过程将因CA与另一CA不同而有所不同，Cloudera建议您参考CA的文档以了解更多详细信息。

Creating Self-Signed Test Certificates

　　Important: Cloudera strongly recommends against the use of self-signed certificates in production clusters.

　　翻译就是：

　　重要提示：Cloudera强烈建议不要在生产群集中使用自签名证书。

　　It is also possible to create your own test certificates. These certificates are typically self-signed; that is, they are signed by your own private key, rather than that of an external CA. Such test certificates are useful during testing and bringup of a cluster.

　　To generate a self-signed certificate, use keytool -genkeypair. (In addition to creating a public/private key pair, this command wraps the public key into a self-signed certificate.) For example, the following command creates a self-signed test certificate for the host node1.example.com, and stores it in a keystore named node1.keystore:

　　翻译就是：

　　也可以创建自己的测试证书。这些证书通常是自签名的; 也就是说，它们由您自己的私钥签名，而不是外部CA的密钥。这样的测试证书在测试和启动集群期间很有用。

　　要生成自签名证书，请使用keytool -genkeypair。（除了创建公钥/私钥对之外，此命令将公钥包装到自签名证书中。）例如，以下命令为主机　　node1.example.com创建一个自签名测试证书，并存储它在密钥库中）

$ keytool -genkeypair -keystore node1.keystore -keyalg RSA \
-alias node1 -dname "CN=node1.example.com O=Hadoop" \
-storepass changeme -keypass changeme

　　Set -keypass to the same value as -storepass. Cloudera Manager assumes that the same password is used to access both the key and the keystore, and therefore, does not support separate values for -keypass and -storepass.

　　翻译就是：　　

　　将-keypass设置为与-storepass相同的值。 Cloudera Manager假定使用相同的密码访问密钥库和密钥库，因此不支持-keypass和-storepass的单独值。

　　然后接着　

二、Creating Truststores（创建信任域）

　　The steps involved in preparing the truststores to be used in your deployment depend on whether you have decided to use the default Java truststore, or to create custom truststores:

If you are using the default truststore, you may need to add CA certificates (or certificate chains) to the truststore, or delete them from the truststore.
If you are creating custom truststores, you will need to build the truststores by importing trusted certificates into new truststores. The trusted certificates can be CA certificates (typically downloaded from the CA's website), or self-signed certificates that you have created.

　　As shown in the examples below, when creating a truststore you must select a password. All truststore passwords for a given service must be the same. In practice, this restriction rarely comes into play, since it is only relevant when you wish to create distinct custom truststores for each host.

　　The following sections provide examples of the steps required for several common scenarios:

　　翻译就是：

　　准备在部署中使用的信任库所涉及的步骤取决于您是否决定使用默认Java信任库，还是创建自定义信任库：
　　如果您使用默认的信任库，则可能需要将CA证书（或证书链）添加到信任库中，或者从信任库中删除它们。
　　如果要创建自定义信任库，则需要通过将受信任的证书导入新的信任库来构建信任库。受信任的证书可以是CA证书（通常从CA的网站下载）或您创建的自签名证书。
　　如下面的示例所示，创建信任库时，您必须选择一个密码。给定服务的所有信任密码必须相同。实际上，这种限制很少发挥作用，因为只有当您希望为每个主机创建不同的自定义信任库时，这种限制才是有用的。

　　以下部分提供了几个常见场景所需步骤的示例：

Example 1: Adding a CA Certificate to the alternative Default Truststore（将CA证书添加到备用的默认信任域）

　　In this example, we assume that you have chosen to use the default Java truststore, but have obtained a certificate from a CA not included in the truststore. (This situation can also arise if the CA that issued your certificate has an entry in the default truststore, but the particular certificate product you purchased requires an alternate CA certificate chain.)

Locate the default truststore on your system. The default truststore is located in the $JAVA_HOME/jre/lib/security/cacerts file. This contains the default CA information shipped with the JDK. Create an alternate default file called jssecacerts in the same location as the cacerts file. You can now safely append CA certificates for any private or public CAs not present in the default cacerts file, while keeping the original file intact.
The alternate file will always be read unless the javax.net.ssl.trustStore flag is set in the arguments for the startup of the java process.

For our example, we will be following this recommendation by copying the default cacerts file into the new jssecacerts file.

　　翻译就是：

　　在此示例中，我们假设您已选择使用默认的Java信任库，但已从不包含在信任库中的CA获取证书。（如果发出证书的CA在默认信任库中有一个条目，但是您购买的特定证书产品需要备用CA证书链，则也可能会出现这种情况。）

　　找到系统上的默认信任库。默认的truststore位于$ JAVA_HOME / jre / lib / security / cacerts文件中。这包含JDK附带的默认CA信息。在与cacerts文件相同的位置创建一个名为jssecacerts的备用默认文件。您现在可以安全地为默认cacerts文件中不存在的任何私有或公共CA安全地附加CA证书，同时保持原始文件不变。
　　除非在java进程启动的参数中设置javax.net.ssl.trustStore标志，否则备用文件将始终被读取。

　　对于我们的示例，我们将通过将默认cacerts文件复制到新的jssecacerts文件中来执行此建议。

$ cp $JAVA_HOME/jre/lib/security/cacerts \
  $JAVA_HOME/jre/lib/security/jssecacerts







If you use a copy of the cacerts file, remember the default keystore password is changeit.

翻译就是：

　　如果您使用cacerts文件的副本，请记住默认密钥库密码为changeit。

　　　2、Import the CA certificate into the default truststore. Assuming that the file myCA-root.cer contains the CA’s certificate, which you have previously downloaded from the CA’s web site, the following command imports this certificate into the alternative default truststore。

　　Note: Test the trust relationship before you import any intermediary CA certificates. Trust should be derived from the root CA only. Import intermediary CA certificates only if necessary.

　　翻译就是：

　　将CA证书导入默认的信任库。假设文件myCA-root.cer包含先前从CA的网站下载的CA的证书，则以下命令会将此证书导入备用的默认信任库。

　　注意：在导入任何中间CA证书之前，请先测试信任关系。信任应仅从根CA派生。仅在必要时才导入中间CA证书。

$ keytool -importcert -file myCA-root.cer -alias myCA \
-keystore /usr/java/default/jre/lib/security/jssecacerts \
-storepass changeit

　　When you give this command, you will be prompted to confirm that you trust the certificate. Be sure to verify that the certificate is genuine before importing it.

　　Important: Any updates you make to the default truststore must be made on all hosts in the cluster.

　　翻译就是：

　　当您给出此命令时，系统将提示您确认您信任该证书。在导入证书之前，请确保证书是真实的。
　　重要提示：必须对集群中的所有主机进行对默认信任库的任何更新。

Example 2: Creating a Custom Truststore Containing a Single CA Certificate Chain（创建包含单个CA证书链的自定义信任域）

　　In this example, we demonstrate how to use keytool to create a custom truststore. We assume all certificates were issued by a single CA, so a truststore containing the certificate chain for that CA will serve for all hosts in the cluster.

　　Our example certificate chain consists of a root certificate and a single intermediate certificate. We assume that you have downloaded these and saved them in the files myCA-root.cer and myCA-intermediate.cer(respectively). The steps below show the commands needed to build a custom truststore containing the root and intermediate certificates.

　　示例2：
在这个例子中，我们演示了如何使用keytool来创建自定义的信任库。我们假设所有证书都由单个CA颁发，因此包含该CA的证书链的信任库将为集群中的所有主机提供服务。

　　我们的示例证书链由根证书和单个中间证书组成。我们假设您已经下载这些文件并将它们保存在文件myCA-root.cer和myCA-intermediate.cer（分别）中。以下步骤显示构建包含根证书和中间证书的自定义信任库所需的命令。

Import the root certificate and create the truststore:
```
$ keytool -importcert -keystore my.truststore -alias myCA-root \
-storepass trustchangeme -file myCA-root.cer
```
You will be prompted to confirm that the root certificate is trustworthy. Be sure to verify that the certificate is genuine before you import it.

　　翻译就是：

　　导入根证书并创建信任库：

　　系统将提示您确认根证书是否值得信赖。在导入证书之前，请确保证书是真实的。

　　2、 Import the intermediate certificate into the truststore created in Step 1:

$ keytool -importcert -keystore my.truststore \
-alias myCA-intermediate -storepass trustchangeme \
-file myCA-intermediate.cer

　　翻译就是：将中间证书导入到步骤1中创建的信任库：

Example 3: Creating a Custom Truststore Containing Self-Signed Test Certificates（创建包含自签名测试证书的自定义信任商店）

　　Important: Cloudera strongly recommends against the use of self-signed certificates in production clusters.

　　This example is particularly relevant when setting up a test cluster. We assume that you have generated a set of self-signed test certificates for the hosts in the cluster, and wish to create a single truststore that can be deployed on all hosts. Because the certificates are self-signed, we cannot simply construct a truststore containing a single certificate chain, as in the previous example. When a client receives a self-signed certificate from a server during the SSL handshake, it must be able to find the server’s certificate in the truststore, since no other signing certificate exists to establish trust. Therefore, the truststore must contain all the test certificates.

　　We assume that the test certificates reside in keystores named node1.keystore … node100.keystore, which were created following the steps described in Creating Self-Signed Test Certificates.

　　翻译就是：

　　重要提示：Cloudera强烈建议不要在生产群集中使用自签名证书。
　　此示例在设置测试集群时特别有用。我们假设您为集群中的主机生成了一组自签名测试证书，并希望创建一个可以在所有主机上部署的信任库。因为证书是自签名的，所以我们不能简单地构建一个包含单个证书链的信任库，如前面的例子所示。当客户端在SSL握手期间从服务器接收到自签名证书时，它必须能够在信任库中找到服务器的证书，因为不存在建立信任的其他签名证书。因此，信任库必须包含所有测试证书。

　　我们假定测试证书位于名为node1.keystore ... node100.keystore的密钥库中，它们是按照创建自签名测试证书中描述的步骤创建的。

Export the test certificate for node1.example.com:

$ keytool -exportcert -keystore node1.keystore -alias node1 \
-storepass changeme -file node1.cer

Import the test certificate into the truststore:
```
keytool -importcert -keystore my.truststore -alias node1 \
-storepass trustchangeme -file node1.cer -noprompt
```
Here we specify the -noprompt option to suppress the prompt asking you to confirm that the certificate is trustworthy. Since you created the certificate yourself, this confirmation is unnecessary.
Repeat Steps 1 and 2 for node2.keystore … node100.keystore.

　　翻译就是：

　　1、导出node1.example.com的测试证书：

　　2、将测试证书导入到信任库中：

　　　　这里我们指定-noprompt选项来禁止提示，要求您确认证书是值得信赖的。由于你自己创建了证书，所以这个确认是不必要的。

　　3、对node2.keystore ... node100.keystore重复步骤1和2。

方法二：解决办法

　　参考：

　　https://www.pythian.com/blog/using-ansible-to-secure-cloudera-manager-installation-on-a-hadoop-cluster/

　　Building a secure Hadoop cluster requires protecting a number of services that comprise the Hadoop infrastructure. If you are using CDH distribution, then Cloudera Manager (CM) is one of the components that needs to be secured. CM consists of Server and Agents running on all cluster machines and web UI. CM provides you with 3 levels of security for its communication (good/better/best):

　　翻译就是：

　　构建安全的Hadoop集群需要保护构成Hadoop基础架构的许多服务。如果您正在使用CDH分发，那么Cloudera Manager（CM）是需要保护的组件之一。 CM由在所有集群机器和Web UI上运行的服务器和代理组成。 CM为您提供3种安全级别的通信（好/更好/最好）：

Transport Layer Security (TLS) Encryption for Cloudera Manager
TLS Authentication of Server to Agent and Users
TLS Authentication of Agents to Server

Cloudera Manager的传输层安全（TLS）加密
服务器到代理和用户的TLS认证
代理服务器的TLS认证

　　There is a good step-by-step guide in CM’s documentation, and it’s easy to follow for one server. But what about when you have hundreds of them? There are different approaches to the problem of managing a server’s configuration to scale, but I’d like to focus on Ansible, which is a neat framework for parallel command execution and complex rollouts. And it’s written in Python! Ansible is easy to install and requires only a couple of Python libraries on a “master” node and nothing other than python2.6 on “slaves”. You don’t have to setup ssh-keys and configure passwordless access across all machines, which is important when talking about security. To give you and idea of how Ansible command looks like, here is how you can restart CM Agents on all Hadoop data nodes:

　　翻译就是：　　

　　CM的文档中有一个很好的分步指南，一个服务器很容易遵循。但是，当你有数百人的时候呢？管理服务器配置以扩展的问题有不同的方法，但我想关注Ansible，这是并行命令执行和复杂推出的整洁框架。它是用Python编写的！可安装易于安装，只需要“主”节点上的几个Python库，而在“从站”上只需要python2.6。您不必在所有机器上设置ssh-key并配置无密码访问，这在谈论安全性时非常重要。为了让您了解Ansible命令的样子，以下是如何在所有Hadoop数据节点上重新启动CM Agent：

ansible hadoop_data_nodes -m service\ 
 -a "name=cloudera-scm-agent state=restarted"  -k --sudo

　　This command will read a list of hosts from your /etc/ansible/hosts file or any other file specified by ANSIBLE_HOSTS environment variable. It will find a section [hadoop_data_nodes] in the hosts file (which has ini-like structure) and will execute any given command for all servers in this section. Ansible will ask you for both ssh and sudo passwords only once and will use them to execute commands on target servers. Ansible relies on modules (specified by -m option) to perform specific tasks, like restarting services, executing shell commands, or manipulating text files. A full list of modules can be found here. You can write your own, of course, if you need.

　　翻译就是：

　　此命令将从/ etc / ansible / hosts文件或由ANSIBLE_HOSTS环境变量指定的任何其他文件读取主机列表。它将在hosts文件中找到一个[hadoop_data_nodes]部分（具有类似于ini的结构），并且将在本节中为所有服务器执行任何给定的命令。 Ansible会询问你们ssh和sudo密码只有一次，并使用它们在目标服务器上执行命令。可靠依赖于模块（由-m选项指定）来执行特定任务，例如重新启动服务，执行shell命令或操作文本文件。可以在这里找到完整的模块列表。当然，如果你需要，你可以自己写。

　　Below is the step-by-step guide on configuring all 3 levels of TLS for CM. In some aspects, it repeats steps from CM’s documentation guide, but there are some important nuances that I have discovered. This guide is also scalable: You can apply it with very little modifications to clusters of any size.

　　翻译就是：以下是为CM配置所有3级TLS的分步指南。在某些方面，它重复CM的文档指南中的步骤，但我发现了一些重要的细微差别。本指南也是可扩展的：您可以对任何大小的群集进行很少的修改来应用它。

Level I、 TLS Encryption for Cloudera Manager（翻译就是Cloudera Manager的TLS加密）

　　The following steps assume that you have installed Ansible on the same server as your CM Server and properly configured it to access all nodes in your cluster.

　　翻译就是：以下步骤假定您已在与CM服务器相同的服务器上安装了Ansible，并将其正确配置为访问集群中的所有节点。

1、Create a directory for CM keystore:

　　翻译就是：为CM密钥库创建一个目录

mkdir /etc/cloudera-scm-server/keystore

2、Generate a certificate for CM. You will be prompted for a new Keystore password. Also make sure that the CN field you will be asked to specify matches the CM Server hostname. It’s not *your* name!

　　翻译就是：为CM生成证书。系统将提示您输入新的密码库密码。还要确保要求您指定的CN字段与CM Server主机名匹配。这不是你*的名字！

keytool -validity 1095 -keystore\
 /etc/cloudera-scm-server/keystore/scm-keystore -alias jetty\
  -genkeypair -keyalg RSA

　　note -validity option, new certificate will be valid for 3 years in this case

　　翻译就是：note -validity选项，新证书在这种情况下将有效3年

3、Restrict permissions to the Keystore:

　　翻译就是：限制对Keystore的权限：

chown -R cloudera-scm:cloudera-scm /etc/cloudera-scm-server/keystore/
chmod o-r /etc/cloudera-scm-server/keystore/scm-keystore

4 、Enable TLS Encryptions for Agents and provide path and password to the Keystore in CM web UI. You can refer to CM’s documentation on how to do this.、|\

　　翻译就是：启用代理的TLS加密，并在CM Web UI中为密钥库提供路径和密码。 您可以参考CM关于如何执行此操作的文档。







5、Next, we need to update CM Agent configuration files to set use_tls=1 option. For this, Ansible lineinfile module can be used:



翻译就是：接下来，我们需要更新CM Agent配置文件以设置use_tls = 1选项。 为此，可以使用Ansible lineinfile模块：

ansible hadoop -m lineinfile -a\
  "dest=/etc/cloudera-scm-agent/config.ini state=present regexp='use_tls.*'\
     line='use_tls=1'" -k -K

　　There are several assumptions made here. First of all, your Ansible host list has a [hadoop] section in it, which covers all Hadoop cluster nodes or has sections like [hadoop_namenodes], [hadoop_datanodes], etc. Ansible can recognize patterns. Second, your template for CM Agnet config.ini files has all security related options in it, but they are commented out. Maintaining a template for all configuration files is a good idea, and Ansible can help you here as well, but it’s beyond the scope of this guide.

　　翻译就是：这里有几个假设。首先，您的可选主机列表中有一个[hadoop]部分，其中包含所有Hadoop集群节点，或包含[hadoop_namenodes]，[hadoop_datanodes]等部分。可以使用可识别的模式。其次，您的CM Agnet config.ini文件的模板中包含所有与安全性相关的选项，但它们被注释掉。维护所有配置文件的模板是一个好主意，Ansible也可以帮助您，但这超出了本指南的范围。

　　6、Restart CM Server (again, I assume that you run Ansible commands from the same machine):

　　翻译就是：重新启动CM服务器（再次，我假设您从同一台机器运行可执行的命令）：

sudo /etc/init.d/cloudera-scm-agent restart

　　7、Restart CM Agents and check their health:

　　翻译就是：重新启动CM代理并检查其身体状况：

ansible hadoop -m service -a\
  'name=cloudera-scm-agent state=restarted' -k -K;

ansible hadoop -m shell -a\
  "tail /var/log/cloudera-scm-agent/cloudera-scm-agent.log" -k -K





You have already seen an example with service Ansible module. The new module used here is shell. It allows you to run arbitrary shell commands. It’s a good idea to verify that all agents started fine, so you can check the log files on all servers.



翻译就是：

您已经看到了一个服务Ansible模块的例子。 这里使用的新模块是shell。 它允许您运行任意shell命令。 验证所有代理程序是否正常启动是一个好主意，因此您可以检查所有服务器上的日志文件。

　　If all is fine, at this point you have encrypted communication between CM Server and CM Agents.

　　翻译就是：如果一切正常，此时您将在CM Server和CM Agent之间进行加密通信。

Level II、 TLS Authentication of Server to Agent and Users（服务器到代理和用户的TLS认证）

　　At this level, you will force CM Agents to check a certificate for CM Server to make sure they are talking to the right machine.

　　翻译就是： 在这个级别上，您将强制CM Agent检查CM Server的证书，以确保它们与正确的机器通话。

Export CM Server Certificate from Keystore and convert it to .der format: （翻译就是从密钥库导出CM服务器证书并将其转换为.der格式：）

keytool -exportcert -keystore\
  /etc/cloudera-scm-server/keystore/scm-keystore -alias jetty\
  -file scm-server.der
openssl x509 -out scm-server.pem -in scm-server.der -inform der

Create new cert dirs on all agent servers: （翻译就是在所有代理服务器上创建新的证书）
```
ansible hadoop -m shell -a "mkdir /etc/cloudera-scm-agent/cert" -k -K
```

Copy server cert to agents: （将服务器证书复制到代理）

ansible hadoop -m copy -a "src=/tmp/scm-server.pem
      dest=/etc/cloudera-scm-agent/cert/ owner=root group=root" -k -K

Change very_cert_file option in agent’s config.ini: （在代理的config.ini中更改very_cert_file选项）

ansible hadoop -m lineinfile -a "dest=/etc/cloudera-scm-agent/config.ini 
  state=present regexp='verify_cert_file' 
  line='verify_cert_file=/etc/cloudera-scm-agent/cert/scm-server.pem'" -k -K

Enabled Use TLS Encryption for Admin Console in CM Web UI （启用对CM Web UI中的管理控制台使用TLS加密）
Restart CM server and Agents. See Steps 6-7 in Level I. （重新启动CM服务器和代理。见第一级步骤6-7。）

Level III、 TLS Authentication of Agents to Server

　　This is similar to the previous level, but requires certificates to be generated for all agents so they can authenticate to the CM Server. Here I describe an approach with self-signed certificates, which is fine for development or POC clusters. For production clusters, you may need to comply with your organisation’s standards and obtain properly signed certificates.

　　翻译就是：　　

　　这与上一级相似，但需要为所有代理生成证书，以便他们可以向CM Server进行身份验证。我在这里描述一种具有自签名证书的方法，这对于开发或POC集群是很好的。对于生产集群，您可能需要遵守组织的标准并获得正确签名的证书。

Generate password for Agent keys and copy it to all agents machines:

ansible hadoop -m shell -a\
 'echo PASSWORD > /etc/cloudera-scm-agent/cert/agent_cert.pwd' -k -K
ansible hadoop -m shell -a\
  'chmod o-rx /etc/cloudera-scm-agent/cert/agent_cert.pwd' -k -K

The next step is to generate a private key and certificate for each CM Agent. To automate this task I wrote a quick script — https://github.com/dazbur/morecerts. It takes a list of hosts in a plain text file and produces a private key and certificate for each host. Resulting files will have agent_HOSTNAME.key and agent_HOSTNAME.pem format. It also takes care of providing Distinguished Name options for keys-generating commands:
- Create a text file with list of Agent’s IP adress. You can just copy it from Ansible hosts lists.
- Create agent key password and put it into a text file.
- Generate keys:
```
./morecerts.py -f agentiplist.file -p agent.pass.file gencerts
```
- Add keys to CM Server keystore:
```
sudo ./morecerts.py -f agentiplist.file -p agent.pass.file -k\
 /etc/cloudera-scm-server/keystore/scm-keystore\
    -w KEYSTORE_PASSOWORD addtokeystore
```
Copy keys and certs to agent machines (assuming you have generated keys in your home directory):
```
ansible hadoop -m copy -a "src=~/agent_$inventory_hostname.key
 dest=/etc/cloudera-scm-agent/cert/agent.key owner=root group=root"  -k -K
ansible hadoop -m copy -a "src=~/agent_$inventory_hostname.pem
 dest=/etc/cloudera-scm-agent/cert/agent.pem owner=root group=root"  -k -K
```
Here you can see another nice trick Ansible can do: $inventory_hostname variable corresponds to the server on which command will be executed. This allows you to copy specific files to specific servers.

Update agent config.ini files to set client_key_file, client_keypw_file, and client_cert_fileoptions:

ansible hadoop -m lineinfile -a "dest=/etc/cloudera-scm-agent/config.ini 
 state=present regexp='client_key_file.*' 
  line='client_key_file=/etc/cloudera-scm-agent/cert/agent.key'" -k -K

ansible hadoop -m lineinfile -a "dest=/etc/cloudera-scm-agent/config.ini 
  state=present regexp='client_keypw_file.*' 
  line='client_keypw_file=/etc/cloudera-scm-agent/cert/agent_cert.pwd'" -k -K

ansible hadoop -m lineinfile -a "dest=/etc/cloudera-scm-agent/config.ini 
  state=present regexp='client_cert_file.*' 
  line='client_cert_file=/etc/cloudera-scm-agent/cert/agent.pem'" -k -K

Update CM’s configuration via Web UI to set Use TLS Authentication of Agents to Server, Path to Truststore, and Truststore Password. Truststore is the same as Keystore in our case.
Restart CM Server and Agents.

You are done now! Ansible can also wrap individual playbooks, so you can execute them on demand later. In general, I found Ansible to be a great tool to manage and execute commands on many servers, and it is definitely worth exploring.

本文转自大数据躺过的坑博客园博客，原文链接：http://www.cnblogs.com/zlslch/p/7130563.html，如需转载请自行联系原作者

cloudera-scm-server启动时出现Caused by: java.io.FileNotFoundException: /var/lib/cloudera-scm-server/.keystore (No such file or directory)问题解决方法（图文详解）

Creating Java Keystores and Truststores（创建Java密钥库和信任库）

Security Considerations for Keystores and Truststores

（密钥库和信任域的安全注意事项）

Creating Keystores（床架密钥库）

Creating Certificates（创建证书）

一、Creating Certificates（创建证书）

Using Keytool（使用keytool）

Using OpenSSL（使用OpenSSL）

Obtaining a Production Certificate from a Commercial CA（从商业CA获得生产证书）

Creating Self-Signed Test Certificates

二、Creating Truststores（创建信任域）

Example 1: Adding a CA Certificate to the alternative Default Truststore（将CA证书添加到备用的默认信任域）

Example 2: Creating a Custom Truststore Containing a Single CA Certificate Chain（创建包含单个CA证书链的自定义信任域）

Example 3: Creating a Custom Truststore Containing Self-Signed Test Certificates（创建包含自签名测试证书的自定义信任商店）

Level I、 TLS Encryption for Cloudera Manager（翻译就是Cloudera Manager的TLS加密）

Level II、 TLS Authentication of Server to Agent and Users（服务器到代理和用户的TLS认证）

Level III、 TLS Authentication of Agents to Server

热门文章

最新文章

相关课程

相关电子书

相关实验场景

热门

活动广场

任务中心

开发者评测

高校计划

乘风者计划

训练营

阿里云MVP

话题

直播

下载

镜像站

技术资料

插件

cloudera-scm-server启动时出现Caused by: java.io.FileNotFoundException: /var/lib/cloudera-scm-server/.keystore (No such file or directory)问题解决方法（图文详解）

Creating Java Keystores and Truststores（创建Java密钥库和信任库）

Security Considerations for Keystores and Truststores

（密钥库和信任域的安全注意事项）

Creating Keystores（床架密钥库）

Creating Certificates（创建证书）

一、Creating Certificates（创建证书）

Using Keytool（使用keytool）

Using OpenSSL（使用OpenSSL）

Obtaining a Production Certificate from a Commercial CA（从商业CA获得生产证书）

Creating Self-Signed Test Certificates

二、Creating Truststores（创建信任域）

Example 1: Adding a CA Certificate to the alternative Default Truststore（将CA证书添加到备用的默认信任域）

Example 2: Creating a Custom Truststore Containing a Single CA Certificate Chain（创建包含单个CA证书链的自定义信任域）

Example 3: Creating a Custom Truststore Containing Self-Signed Test Certificates（创建包含自签名测试证书的自定义信任商店）

Level I、 TLS Encryption for Cloudera Manager（翻译就是Cloudera Manager的TLS加密）

Level II、 TLS Authentication of Server to Agent and Users（服务器到代理和用户的TLS认证）

Level III、 TLS Authentication of Agents to Server

热门文章

最新文章

相关课程

相关电子书

相关实验场景

　　（密钥库和信任域的安全注意事项）