一、环境准备

1、服务器资源分配

(资料图片)

2、禁用selinux

> vi /etc/selinux/config SELINUX=disabled

3、禁用防火墙

> systemctl disable firewalld> systemctl stop firewalld

4、安装JDK

> tar -zxvf jdk-8u102-linux-x64.tar.gz -C /usr/local/> vi /etc/profile.d/java.shexport JAVA_HOME=/usr/local/jdk1.8.0_102 export JRE_HOME=$JAVA_HOME/jre export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$JAVA_HOME:$PATH> source /etc/profile

二、部署Kerberos

1、安装Kerberos，通过在线安装的方式安装KDC客户端

> yum -y install krb5-server krb5-libs krb5-workstation

2、配置krb5.conf

> vi /etc/krb5.conf[libdefaults]default_realm = NIFI.COM  #此处需要进行配置[realms]NIFI.COM ={      kdc = server   #此处配置的为主机名server    admin_server = server  #同上, 配置的为主机名server}[domain_realm].nifi.com = NIFI.COM  #此处配置与前面配置[realms]保持一致nifi.com = NIFI.COM

Realm参数自己设定但要一致kdc=主机ipadmin_server=主机ipKdc为kdc位置，格式为 主机：端口（可用默认）（主机可以是主机名或ip）

> scp /etc/krb5.conf 192.168.0.12:/etc/> scp /etc/krb5.conf 192.168.0.13:/etc/

3、配置kdc.conf

> vi /var/kerberos/krb5kdc/kdc.conf

[realms] NIFI.COM = {   #master_key_type = aes256-cts   acl_file = /var/kerberos/krb5kdc/kadm5.acl   dict_file = /usr/share/dict/words   admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab   supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal }此处为NIFI.COM与/etc/krb5.conf中的配置保持一致

4、配置kadm5.acl

> vi /var/kerberos/krb5kdc/kadm5.acl */admin@NIFI.COM *

5、创建Kerberos数据库

> kdb5_util create -r NIFI.COM -sEnter KDC database master key: 123456a?Re-enter KDC database master key to verify: 123456a?

当长时间Loading random data时，另开一个控制台使用如下命令

> cat /dev/sda > /dev/urandom

6、添加用户

> kadmin.localkadmin.local: addprinc superadminEnter password for principal "admin/admin@NIFI.COM": superadminRe-enter password for principal "admin/admin@NIFI.COM":superadminkadmin.local: addprinc testEnter password for principal "admin/admin@NIFI.COM": 123456a?Re-enter password for principal "admin/admin@NIFI.COM":123456a?kadmin.local: exit

7、启动Kerberos

> systemctl start krb5kdc> systemctl start kadmin> systemctl enable krb5kdc> systemctl enable kadmin

8、 添加服务主体并导出keytab

> kadmin.local> addprinc -randkey test/NIFI> ktadd -k /opt/test-NIFI.keytab test/NIFI> q

9、创建web登陆用户创建login/server@NIFI.COM作为管理员账户，自定义密码为hadoop

kadmin.local -q "addprinc login/server"

使用kinit命令来检测创建的用户是否成功(需要输入密码)

kinit login/server@NIFI.COM

三、证书生成

利用Toolkit工具生成证书，Nifi官方提供了一个工具用户生成相关的证书，参考文档 Apache NiFi Admin Guide 的 TLS Generation Toolkit模块:1、下载与安装下载地址：http://nifi.apache.org/download.html下载后解压安装nifi-toolkit-1.16.0-bin.zip (或tar.gz格式，自己选择最新版本)，配置好环境变量方便使用。2、生成相关证书为server节点生成密钥库，truststore，nifi.properties以及具有给定DN的客户端证书。

bin/tls-toolkit.sh standalone -O -n "172.16.0.11" -C "CN=superadmin, OU=NIFI" --subjectAlternativeNames "172.16.0.11,0.0.0.0" -o "target1"

其中-c, -certificateAuthorityHostname 为Hostname of NiFi Certificate Authority。-C,–clientCertDn 生成适合在指定DN的浏览器中使用的客户端证书（可以指定多次）-n，-hostnames要生成证书的主机名列表(以逗号分隔)，可指定多次，支持范围和实例模式。-o, -outputDirectory 指定生成证书的路径-O,–isOverwrite 覆盖现有主机输出–subjectAlternativeNames 在证书中用作主题备用名称的域的逗号分隔列表3、分发证书

把node1目录下的文件拷贝到node1节点下nifi的安装目录的conf文件下，重复的文件选择覆盖，同样把node2目录下的文件拷贝到node2节点下nifi的安装目录的conf文件下，把node3目录下的文件拷贝到node3节点下nifi的安装目录的conf文件下。

scp ./192.168.0.11/* /usr/local/nifi-1.16.0/conf/scp ./192.168.0.12/* root@192.168.0.12:/usr/local/nifi-1.16.0/conf/scp ./192.168.0.13/* root@192.168.0.13:/usr/local/nifi-1.16.0/conf/

四、NIFI配置

1、 配置nifi常用参数采用nifi-toolkit工具生成的证书，在生成证书的过程中已经生了一个nifi.properties配置文件，把不同目录下的证书文件拷贝到不同节点nifi的conf目录下，重复的文件选择覆盖然后修改nifi.properties中的一些参数

nifi.state.management.embedded.zookeeper.start=truenifi.cluster.protocol.is.secure=truenifi.cluster.is.node=true nifi.zookeeper.connect.string=192.168.0.11:2181,192.168.0.12:2181,192.168.0.13:2181

采用了nifi中的自带zookeeper，设置三个节点的zookeeper.修改配置了zookeeper节点的nifi安装目录下的/conf/state-management.xml

192.168.0.11:2181,192.168.0.12:2181,192.168.0.13:2181

修改每个节点的/conf/zookeeper.properties

server.1=192.168.0.11:2888:3888;2181server.2=192.168.0.12:2888:3888;2181server.3=192.168.0.13:2888:3888;2181

创建myid文件,主要根据zookeeper.properties文件的的dataDir=./state/zookeeper参数确定位置在此文件夹下（没有则创建）不同节点分别执行

echo 1 > ./state/zookeeper/myidecho 2 > ./state/zookeeper/myidecho 3 > ./state/zookeeper/myid

2、配置kerberos参数

vi nifi.propertiesnifi.remote.input.secure=truenifi.security.user.login.identity.provider=kerberos-providernifi.security.user.authorizer=file-providernifi.sensitive.props.key=123456789012 #所有选项都需要至少 12 个字符的密码（nifi.sensitive.props.key值）# kerberos # nifi.kerberos.krb5.file=/etc/krb5.confnifi.kerberos.service.principal=test/NIFI@NIFI.COM #配置的服务主体nifi.kerberos.service.keytab.location=/opt/test-NIFI.keytab #配置服务主体的keytab

把配置服务主体的/opt/test-NIFI.keytab拷贝到每个节点的/opt目录下

scp test-NIFI.keytab root@192.168.0.12:/opt/scp test-NIFI.keytab root@192.168.0.13:/opt/

3、 配置nifi登陆信息

vi login-identity-providers.xml

   kerberos-provider   org.apache.nifi.kerberos.KerberosProvider   NIFI.COM   12 hours 

放开kerberos-provide的配置，把相关注释部分取消，然后配置成你前面设置的Realm。4、配置用于登陆的用户信息

vi authorizers.xml

        file-provider        org.apache.nifi.authorization.FileAuthorizer        ./conf/authorizations.xml        ./conf/users.xml                superadmin@NIFI.COM                       CN=192.168.0.11, OU=NIFI        CN=192.168.0.12, OU=NIFI        CN=192.168.0.13, OU=NIFI 

5、配置JVM修改 bootstrap.conf，默认2048太小

> java.arg.2=-Xms10240m> java.arg.3=-Xmx10240m

6、 启动服务，登陆web页面配置完成后，每个节点启动nifi：nifi.sh start，启动完成后用jps查看相关进程，或用curl命令来验证nifi是否启动：curl --insecure https://192.168.0.11:9443/nifi每个节点都能登陆访问web页面

Hosts:设置的ip

Port: 9443(默认，或者你修改的端口)

User: superadmin

Password: ****(前面kerberos配置的用户名和密码)

五、集群搭建过程中出现的问题

1、2019-04-08 08:26:28,522 WARN [Replicate Request Thread-5] o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET /nifi-api/flow/current-user to x.x.x.x:9443 due to

解决办法： 使用 tls-toolkit 重新创建证书，需要指定subjectAlternativeNames

2019-04-08 08:26:28,522 WARN [Replicate Request Thread-5] o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET /nifi-api/flow/current-user to x.x.x.x:9443 due to javax.net.ssl.SSLPeerUnverifiedException: Hostname x.x.x.x not verified: certificate: sha256/IiBnANAzEOL~~~qm7wcLzM= DN: CN=x.x.x.x, OU=NIFI subjectAltNames: [x.x.x.x]2019-04-08 08:26:28,523 WARN [Replicate Request Thread-5] o.a.n.c.c.h.r.ThreadPoolRequestReplicatorjavax.net.ssl.SSLPeerUnverifiedException: Hostname x.x.x.x not verified:certificate: sha256/IiBnANAzEOLVg+DipeiOT+fkIDpqtA3h/Nhqm7wcLzM=DN: CN=x.x.x.x, OU=NIFIsubjectAltNames: [x.x.x.x]at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:316)at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:270)at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:162)at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257)at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)at okhttp3.RealCall.execute(RealCall.java:77)at org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:138)at org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:132)at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:647)at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:839)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)

2、当查看或删除队列数据时出现下面错误解决办法：需要对每个节点添加数据读和写的权限

permission error: Insufficient Permissions Node nifi-test-1:8443 is unable to fulfill this request due to: Unable to view the data for Processor with ID b3169c5d-0175-1000-0000-00001734159f. Contact the system administrator. Contact the system administrator.

六、页面展示

登陆页面：

登陆成功：

集群管理页面