104 views

Hadoop运维技术记录

By | 2019年2月22日

Zeppelin开启https过程和Hack内核以以客户为中心的记录。

缘故是这顾客很有趣,该顾客我国分公司的人以便认证内网安全性,从海外找了1个渗透测试项目组对Zeppelin和别的产品进行网络黑客测试,結果发觉Zeppelin关键俩问题,一个是其他网不起作用https,一个是zeppelin里边可以实行shell命令和python语句。我觉得这算不上问题,zeppelin本来就是干这个用的。但是渗入小组不知道zeppelin是干什么的,觉得即便在内网里,执行shell命令能查看电脑操作系统的某些文档是问题,随后发生的事也不说了,并不是我们的问题了。

不过即然他们规定整改,我们也只好相互配合,尽管大家都觉得内部网网站域名加https属于脱了裤子打屁,随后不让zeppelin干他本来应该干的事就更过分了,但由于顾客是招标方,也只好hack源代码了。

因此某一周未用了四个钟头成功全部工作任务。

先记录查询下zeppelin加https浏览,我们有自己的域名证书,所以立即用即可。如果不是域名证书,需要自签发,那麼可以看第二大部分,双向验证步骤。

https第一部分,现有网站域名加上jks:

openssl pkcs12 -export -in xxx.com.crt -inkey xxx.com.key -out xxx.com.pkcs12
keytool -importkeystore -srckeystore xxx.com.pkcs12 -destkeystore xxx.com.jks -srcstoretype pkcs12
https第二部分,自签发证书双向认证加上jks

# 生成root私钥和证书文件。
openssl genrsa -out root.key(pem) 2048 # Generate root key file
openssl req -x509 -new -key root.key(pem) -out root.crt # Generate root cert file
# 创建客户端私钥和证书以及证书请求文件csr
openssl genrsa -out client.key(pem) 2048 # Generate client key file
openssl req -new -key client.key(pem) -out client.csr # Generate client cert request file
openssl x509 -req -in client.csr -CA root.crt -CAkey root.key(pem) -CAcreateserial -days 3650 -out client.crt # Use root cert to generate client cert file
# 生成服务器端私钥,证书和证书请求文件csr
openssl genrsa -out server.key(pem) 2048 # Generate server key file, use in Zeppelin
openssl req -new -key server.key(pem) out server.csr @ Generate server cert request file
openssl x509 -req -in server.csr -CA root.crt -CAkey root.key(pem) -CAcreateserial -days 3650 -out server.crt # Use root cert to generate server cert file
# 生成客户端端jks文件
openssl pkcs12 -export -in client.crt -inkey client.key(pem) -out client.pkcs12 # Package to pkcs12 format, must input a password, you should remember the password
keytool -importkeystore -srckeystore client.pkcs12 -destkeystore client.jks -srcstoretype pkcs12 # The client password you just input at last step
# 生成服务器端jks文件
openssl pkcs12 -export -in server.crt -inkey server.key(pem) -out server.pkcs12 @ Package to pkcs12 format, must input a password, you should remember the password
keytool -importkeystore -srckeystore server.pkcs12 -destkeystore server.jks -srcstoretype pkcs12 # The server password you just input at last step
假如不是需要点对点验证,要是单向自签发,不建立客户端的各种就可以了。

随后找个地把这种文档放过去,再改动zeppelin配置即可。

mkdir -p /etc/zeppelin/conf/ssl
cp server.crt server.jks /etc/zeppelin/conf/ssl
<property>
<name>zeppelin.server.ssl.port</name>
<value>8443</value>
<description>Server ssl port. (used when ssl property is set to true)</description>
</property>
<property>
<name>zeppelin.ssl</name>
<value>true</value>
<description>Should SSL be used by the servers?</description>
</property>
<property>
<name>zeppelin.ssl.client.auth</name>
<value>false</value>
<description>Should client authentication be used for SSL connections?</description>
</property>
<property>
<name>zeppelin.ssl.keystore.path</name>
<value>/etc/zeppelin/conf/ssl/xxx.com.jks</value>
<description>Path to keystore relative to Zeppelin configuration directory</description>
</property>
<property>
<name>zeppelin.ssl.keystore.type</name>
<value>JKS</value>
<description>The format of the given keystore (e.g. JKS or PKCS12)</description>
</property>
<property>
<name>zeppelin.ssl.keystore.password</name>
<value>password which you input on generating server jks step</value>
<description>Keystore password. Can be obfuscated by the Jetty Password tool</description>
</property>

随后反代那边也添加443的ssl证书以及443转8443的upstream即可。

随后是hack zeppelin源代码加入关键词限制,这个的确找了一小会zeppelin发送到执行源代码给interpreter的地方,zeppelin构架较为清晰,但是源代码挺复杂的,采用了很多花朵活儿。例如thrift,interpreter脚本里创建nc监听。随后各个c编译器插件用socket跟interpreter脚本通信,前端angular,后端jetty,还用shiro做验证和授权。回首可以单开好几篇说说zeppelin安装,使用和详尽配置,做这项目基础把zeppelin猜透了。

寻找发送前端开发编写內容给interpreter的java代码,然后用很生硬的方法限制运行命令。具体那个.java文件的名字我不说了,有悬念有惊喜。我不写java,只负责读源代码找到源代码位置,hack的java是同事写的。然后编译,替换jar包,完成。后边改了改配置,事件的渗透测试圆满通过。

static HashSet<String[]> blockedCodeString = new HashSet<>();
static {
blockedCodeString.add(new String[]{“import”, “os”});
blockedCodeString.add(new String[]{“import”, “sys”});
blockedCodeString.add(new String[]{“import”, “subprocess”});
blockedCodeString.add(new String[]{“import”, “pty”});
blockedCodeString.add(new String[]{“import”, “socket”});
blockedCodeString.add(new String[]{“import”, “commands”});
blockedCodeString.add(new String[]{“import”, “paramiko”});
blockedCodeString.add(new String[]{“import”, “pexpect”});
blockedCodeString.add(new String[]{“import”, “BaseHTTPServer”});
blockedCodeString.add(new String[]{“import”, “ConfigParser”});
blockedCodeString.add(new String[]{“import”, “platform”});
blockedCodeString.add(new String[]{“import”, “popen2”});
blockedCodeString.add(new String[]{“import”, “copy”});
blockedCodeString.add(new String[]{“import”, “SocketServer”});
blockedCodeString.add(new String[]{“import”, “sysconfig”});
blockedCodeString.add(new String[]{“import”, “tty”});
blockedCodeString.add(new String[]{“import”, “xmlrpmlib”});
blockedCodeString.add(new String[]{“etc”});
blockedCodeString.add(new String[]{“boot”});
blockedCodeString.add(new String[]{“dev”});
blockedCodeString.add(new String[]{“lib”});
blockedCodeString.add(new String[]{“lib64”});
blockedCodeString.add(new String[]{“lost+found”});
blockedCodeString.add(new String[]{“mnt”});
blockedCodeString.add(new String[]{“proc”});
blockedCodeString.add(new String[]{“root”});
blockedCodeString.add(new String[]{“sbin”});
blockedCodeString.add(new String[]{“selinux”});
blockedCodeString.add(new String[]{“usr”});
blockedCodeString.add(new String[]{“passwd”});
blockedCodeString.add(new String[]{“useradd”});
blockedCodeString.add(new String[]{“userdel”});
blockedCodeString.add(new String[]{“rm”});
blockedCodeString.add(new String[]{“akka “});
blockedCodeString.add(new String[]{“groupadd”});
blockedCodeString.add(new String[]{“groupdel”});
blockedCodeString.add(new String[]{“mkdir”});
blockedCodeString.add(new String[]{“rmdir”});
blockedCodeString.add(new String[]{“ping”});
blockedCodeString.add(new String[]{“nc”});
blockedCodeString.add(new String[]{“telnet”});
blockedCodeString.add(new String[]{“ftp”});
blockedCodeString.add(new String[]{“scp”});
blockedCodeString.add(new String[]{“ssh”});
blockedCodeString.add(new String[]{“ps”});
blockedCodeString.add(new String[]{“hostname”});
blockedCodeString.add(new String[]{“uname”});
blockedCodeString.add(new String[]{“vim”});
blockedCodeString.add(new String[]{“nano”});
blockedCodeString.add(new String[]{“top”});
blockedCodeString.add(new String[]{“cat”});
blockedCodeString.add(new String[]{“more”});
blockedCodeString.add(new String[]{“less”});
blockedCodeString.add(new String[]{“chkconfig”});
blockedCodeString.add(new String[]{“service”});
blockedCodeString.add(new String[]{“netstat”});
blockedCodeString.add(new String[]{“iptables”});
blockedCodeString.add(new String[]{“ip”});
blockedCodeString.add(new String[]{“route “});
blockedCodeString.add(new String[]{“curl”});
blockedCodeString.add(new String[]{“wget”});
blockedCodeString.add(new String[]{“sysctl”});
blockedCodeString.add(new String[]{“touch”});
blockedCodeString.add(new String[]{“scala.sys.process”});
blockedCodeString.add(new String[]{“0.0.0.0”});
blockedCodeString.add(new String[]{“git”});
blockedCodeString.add(new String[]{“svn”});
blockedCodeString.add(new String[]{“hg”});
blockedCodeString.add(new String[]{“cvs”});
blockedCodeString.add(new String[]{“exec”});
blockedCodeString.add(new String[]{“ln”});
blockedCodeString.add(new String[]{“kill”});
blockedCodeString.add(new String[]{“rsync”});
blockedCodeString.add(new String[]{“lsof”});
blockedCodeString.add(new String[]{“crontab”});
blockedCodeString.add(new String[]{“libtool”});
blockedCodeString.add(new String[]{“automake”});
blockedCodeString.add(new String[]{“autoconf”});
blockedCodeString.add(new String[]{“make”});
blockedCodeString.add(new String[]{“gcc”});
blockedCodeString.add(new String[]{“cc”});
}
static boolean allMatch(String aim, String[] checker){
if(checker == null || checker.length < 1){
return false;
}else {
// by default, treat as match, every not match change it
for (String i : checker) {
if (!aim.matches(“.*\\b” + i + “\\b.*”)){
return false;
}
}
return true;
}
}
static String anyMatch(String aim, HashSet<String[]> all) throws Exception{
if(aim.contains(“FUCK P&G”)){
throw  new Exception(“How do you know this ????”);
} else {
for (String[] one : all) {
if (allMatch(aim, one)) {
StringBuilder sb = new StringBuilder();
for (String s : one) {
sb.append(s + ” “);
}
return sb.toString();
}
}
throw new Exception(“No one match”);
}
}

//……此处是个public类
try{
String matchesStrings = anyMatch(st, blockedCodeString);
result = new InterpreterResult(Code.ERROR, “Contains dangerous code : ” + matchesStrings);
}catch (Exception me){ // no match any
scheduler.submit(job);
while (!job.isTerminated()) {
synchronized (jobListener) {
try {
jobListener.wait(1000);
} catch (InterruptedException e) {
logger.info(“Exception in RemoteInterpreterServer while interpret, jobListener.wait”, e);
}
}
}
if (job.getStatus() == Status.ERROR) {
result = new InterpreterResult(Code.ERROR, Job.getStack(job.getException()));
} else {
result = (InterpreterResult) job.getReturn();
// in case of job abort in PENDING status, result can be null
if (result == null) {
result = new InterpreterResult(Code.KEEP_PREVIOUS_RESULT);
}
}
}
//……直到该public类结束

因为顾客有deadline限制,所以迅速定位源代码位置的全过程还是挺有趣的,较为紧张刺激作用,在这个以钟头计算deadline压力下,什么intelliJ, Eclipse都不好使啊,就grep和vi最好用,从寻找到改完,比顾客定的deadline提前了很多小时。

本文转载于:http://win-man.com/faq
本文关键词:云漫源代码咨询  服务器租用 IDC咨询 加速CDN
作者:云漫网络科技有限公司

发表评论

电子邮件地址不会被公开。 必填项已用*标注