安恒网管员手记: 邮件病毒过滤
2004-06-17    刘世伟   
打印自: 安恒公司
地址: HTTP://dtx-clt.anheng.com.cn/news/article.php?articleid=312
安恒网管员手记之邮件病毒过滤




[安恒原创]
转载请注明出处  http://www.anheng.com.cn/news/15/312.html

本文章动态更新 刘世伟

安恒公司用的邮件服务器是在自己的linux服务器上面安装的qmail和vpopmail
从去年以来一直使用一套python编写的qmfilt过滤程序,今天对它的过滤规则修改了一下.
qmfilt的过滤规则是用正则表达式来书写的。
可以完成一般的病毒的过滤。

这条规则可以过滤掉所有带.scr,.pif,.bat扩展名附件的邮件
^Content-Type: application/octet-stream;\s+name=".*?\.(scr|p if|bat)

这条规则可以过滤掉所有的冲击波病毒,因为它们用接收者的域名做hello
^Received: from unknown .HELO (anheng\.com\.cn|watertest\. com\.cn|anheng\.com)

病毒附件都是exe格式或者zip格式的。在邮件中进行base64编码
对于exe格式,直接选取一段编码后的信息做特征就行了,
ZIP的比较复杂,不太容易对付,因为它压缩后的文件是变化的,压缩之前的文件名
会变化,时间会变化,这样压缩后的文件很难找到一个固定的特征码。
今天对病毒的zip附件格式研究了一下,找到一点规律
zip文件会对压缩的内容进行crc32校验,这个校验码就在zip文件的偏移0e-12处。
我就提取了这个zip的校验码作为特征,完成了对病毒的判别。
首先判断是文件开始是PK,然后中间是任意12个字符,再取4个字符,完整的base64编码后的正则表达式是
^UEsD...............(fsOn8|FWhee)
这个是现在流行的I-Worm-lovGate-w病毒的特征码

^UEsD...............jiB3e //netspy-q



2004.06.18 今天学习了一下python语言,对qmfilt做了如下改进:
过滤规则设置增加了2个字段:MinMail,MaxMail 就是限定过滤规则适用的邮件大小,可以大大减轻服务器的负载,
因为改进前的qmfilt是对邮件执行所有的过滤规则,正则表达式执行起来本来就慢吗。而且python是脚本语言
现在可以只针对特定的邮件大小执行特定的规则。大大减少了过滤操作
python语言居然是用缩进来划分程序段的结构的。有创意。
2004.07.20 增加几个病毒特征码

2004.07.21 增加bagle.z 的zip带密码版本病毒特征码,新情况:病毒用密码压缩跳过病毒监测。
2004.07.24 bagle.n.zip
2004.07.25 新的bagle.z病毒
2004.07.27 lovGateAh
2004.07.28 Bagle-ai 


附最新的qmfilt过滤脚本(2004-07-28):

#^名称::动作::关键词::体积下限::体积上限::
Bagle-aiZIP::drop::^UEsD...............jsUfr::43000::47000::
lovgateAHzip::drop::^UEsD...............ilfOs::206000::216000::
Bagle-nZIP::drop::^UEsD...............u02uH::31000::35000::

NetspyqZip::drop::^UEsD...............jiB3e::39000::43000::
NetspyqExe::drop::AAAAAAAAYAAAAA4fug4AtAnNIbgBTM0hV2luZG93cyBQcm9ncmFtDQokUEUAAEwBAwAAAAAA::39000::43000::
dumaruaExe::drop::kJBUaGlzIHByb2dyYW0gbXVzdCBiZSBydW4gdW5kZXIgV2luMzIN::23000::26000::
NetspybZip::drop::^UEsD...............dbrAi::30000::35000::
NetspycZip::drop::^UEsD...............udsW6::32000::36000::
NetspyaZip::drop::^UEsD...............NS0.3::29000::35000::
LovgatwZip::drop::^UEsD...............(fsOn8|FWhee|KQUf7|CLmRX|dYwmI)::168000::180000::
LovgatwADZip::drop::^UEsD...............dYwmI::190000::205000::
LovgatwADExe::drop::AAAA4AAAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1v::195000::205000::
NetspybExe::drop::AAAAAAAAgAAAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4g::28000::33000::
NetspycZip::drop::^UEsD...............iZMYW::32000::37000::
NetspyrZip::drop::^UEsD................cADh::39000::43000::
NetspycExe::drop::^AAAAAAAAuAAAAGsiX1YvQzEFL0MxBS9DMQWsXz8FI0MxBcdcOwU0QzEFL0MwBXBDMQWsS2wF::33000::37000::
LovgatwExe::drop::SIQaDu7u7hsabHdNhB3DY7u7u8Hd1g7u8Hd3cIBB3cIBV7U7Yz1hPVOVOVOVP6nanbW::160000::180000::
LovgatWExe::drop::AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1v::160000::180000::
Bagle-zExe::drop::6hAAAAAAAAAZAENsb3NlSGFuZGxlADIAQ3JlYXRlRmlsZUEAZAFHZXRXaW5kb3dzRGlyZWN0::33000::35000::
CloseMouseExe::drop::AAAA2AAAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1v::38000::42000::
Bagle-zVBS::drop::ZGltIGZpbGVzeXMsIGZpbGV0eHQsIGdldG5hbWUsIHBhdGgsIHRleHRmaWxlLCBpDQp0ZXh0::99000:105000::
Bagle-zExe::drop::AAAAAAAkAAAAKkm3RPtR7NA7UezQO1Hs0DtR7NA7kezQGNYoEBtR7NAEWehQOxHs0AqQbVA::34000::38000::
Bagle-zZIP::drop::^UEsD...............(gfxM6|JBhhq)::30000::36000::



Bagle-zZIP::drop::^UEsD...............C8sAw::43000::46000::
Bagle-zHta::drop::UExJQ0FUSU9OIElEPSJRIiBBUFBMSUNBVElPTk5BTUU9IlEiIEJPUkRFUj0ibm9uZSIgQk9S::95000::105000::
new1::drop::^NEBQAjHN5ikmFxgFWALrKuu9URtowfUiNOTfIZuAgalVe0ITyxMiDqwkXln4gXX7SYP5NGlapNVh::17000::20000::
hello::drop::^Received: from unknown .HELO (126\.com|anheng\.com\.cn|watertest\.com\.cn|di-an\.com\.cn|anheng\.com|wczyxy\.com)::5000::500000::
scr_pif_bat::drop::^Content-Type: (application/octet-stream|audio/x-wav|audio/x-midi);\s+name=".*?\.(scr|pif|bat|sh.|com|cpl)"::::300000::
源代码
#!/usr/local/bin/python --

"""

This is a program by Dru Nelson. You can contact me
and discuss bugs/issues/kudos at dnelson@redwoodsoft.com.

This program acts as a qmail-queue filter. 
If qmail-smtpd receives a piece of email, it normally
sends it to qmail-queue. With a small patch, it can
now send it to this program which will then call 
qmail-queue.

20000513 - DN - Making the initial version: ILOVEYOU
20040621 - liu shiwei V1.3

"""


import os, sys, string, time, traceback, re

Version  = '1.3'
PyVersion  = '1.0'
Logging  = 1
Debug    = 0
QmailD   = 501
LogFile  = None
TestMode = None
Filters  = []
MailEnvelope = ''
MailData     = ''
MaxData = 2000000
MinData = 0
MailLen = 0
ValidActions = {'trap': '', 'drop' : '', 'block' :'','back' :'' }



def openlog():
  global LogFile

  if not Logging:
     return
  try:
    LogFile = os.open('//var/qmail/qmfilt/qm.log', os.O_APPEND|os.O_WRONLY|os.O_CREAT, 0744) 
  except:
    if TestMode:
      print "Can't create /var/log/qmfilt"
    else:
      # Indicate a write error
      sys.exit(53)

def log(message):
  message = time.asctime(time.localtime(time.time())) + " - " + message + "\n"
  if Logging:
    os.write(LogFile, message)
    # Indicate a write error
    #sys.exit(53)
  if TestMode:
    print message


def showFilters():
  
  if len(Filters) == 0:
    print "No filters"
  for f in Filters:
    print "Filter -  %s - Action: %s  Regex: %s" % (f[0], f[1], f[2])

def readFilters():
  global Filters

  try:
    file = open('/var/qmail/control/qmfilt', 'r') 
    configLines = file.readlines()
    file.close()
  except:
    log("Can't read /var/qmail/control/qmfilt")
    if not TestMode:
      # Indicate a 'cannot read config file error'
      sys.exit(53)

  reg = re.compile('(.*)::(.+)::(.+)::(.*)::(.*)::')
  for line in configLines:
    if line[0] == '#':
      continue
    m = reg.match(line)
    if m != None:
      action = string.lower(m.group(2))
      if not ValidActions.has_key(action):
         log("Invalid action in config file [%s] - SKIPPED" %(action))
         continue
      Filters.append( [m.group(1), string.lower(m.group(2)), m.group(3),m.group(4),m.group(5)] )    
def readDescriptor(desc):
  global MailLen
  result = ''
  while 1:
    data = os.read(desc, 16384)
    if (data == ''):
      break 
    result = result + data
  return result

def writeDescriptor(desc, data):
  while data:
    num  = os.write(desc, data)
    data = data[num:]

  os.close(desc)


def filterHits():
  global MailLen
  MailLen=len(MailData)
  for regEx in Filters:
    if Debug==1:   
       log("size-%d,%s:%s,%s" % (MailLen,regEx[0],regEx[3],regEx[4]))
    if (len(regEx[3])>0) and (MailLen < int(regEx[3])):
       if Debug==1:
         log("skip %d<%s" % (MailLen,regEx[3]))
       continue
    if (len(regEx[4])>0) and (MailLen > int(regEx[4])):
       if Debug==1:	    
         log("skip %d>%s" % (MailLen,regEx[4]))
       continue
    if Debug==1:   
       log("run! %s-%s" %( regEx[0],regEx[2]));
    reg = re.compile(regEx[2], re.M|re.I)
    match = reg.search(MailData)
    if match:
      return regEx[1], regEx[0]
  return None,None


def storeInTrap(hit):
  try:
    pid = os.getpid()
    mailFile = os.open('/var/qmail/qmfilt/qmfilt.%s' %(pid), os.O_WRONLY|os.O_CREAT|os.O_EXCL, 0744) 
  except:
    log("Can't create /var/qmail/qmfilt/qmfilt.%s" %(pid))
    # Indicate a write error
    sys.exit(53)

  try:
    (size, inode, size, size, size,size, size, size, size, size) = os.fstat(mailFile)
    os.rename('/var/qmail/qmfilt/qmfilt.%s' %(pid), '/var/qmail/qmfilt/%s.mail' %(inode))
  except:
    log("Can't rename /var/qmail/qmfilt/qmfilt.%s -> /var/qmail/qmfilt/%s.mail" %(pid, inode))
    # Indicate a write error
    sys.exit(53)

  try:
    envFile = os.open('/var/qmail/qmfilt/%s.envelope' %(inode), os.O_WRONLY|os.O_CREAT|os.O_EXCL, 0744) 
    mesgFile = os.open('/var/qmail/qmfilt/%s.qmfilt' %(inode), os.O_WRONLY|os.O_CREAT|os.O_EXCL, 0744) 
    writeDescriptor(mailFile, MailData)
    writeDescriptor(envFile,  MailEnvelope)
    writeDescriptor(mesgFile, "Matched filter [ %s] to file %s" %(hit, inode))
    log("Matched filter [ %s] to file %s" %(hit, inode))
  except:
    log("Can't create/write files into trap")
    # Indicate a write error
    sys.exit(53)

  return 0


def sendToQmailQueue():
  
  # Open a pipe to qmail queue
  fd0 = os.pipe()
  fd1 = os.pipe()
  pid = os.fork()
  if pid < 0:
    log("Error couldn't fork")
    sys.exit(81)
  if pid == 0:
    # This is the child
    os.dup2(fd0[0], 0)
    os.dup2(fd1[0], 1)
    i = 2
    while (i < 64):
      try:
        os.close(i)
      except:
        pass
      i = i + 1
    os.chdir('/var/qmail')
    #time.sleep(10)
    os.execv("bin/qmail-queue", ('bin/qmail-queue',))
    log("Something went wrong")
    sys.exit(127)  # This is only reached on error
  else:
    # This is the parent
    # close the readable descriptors
    os.close(fd0[0])
    os.close(fd1[0])


  # Send the data
  recvHdr = "Received: (anhengmail: V%s); " %(Version)
  recvHdr = recvHdr + time.strftime("%d %b %Y %H:%M:%S", time.gmtime(time.time()))
  recvHdr = recvHdr + " -0000\n"
  os.write(fd0[1], recvHdr)
  writeDescriptor(fd0[1], MailData)
  writeDescriptor(fd1[1], MailEnvelope)

  # Catch the exit code to return 
  result = os.waitpid(pid, 0)[1]
  if PyVersion > '1.5.1':
    if os.WIFEXITED(result):
      return os.WEXITSTATUS(result)
    else:
      log("Didn't exit normally")
      sys.exit(81)
  else:
    return result


def main(argv, stdout, environ):

  global TestMode
  global MailData
  global MailEnvelope
  global PyVersion
  global MaxData
  global MinData
  PyVersion = string.split(sys.version)[0]

  if len(argv) > 1 and argv[1] == '--test':
    TestMode = 1

  os.setuid(QmailD)

  # Insure Environment is OK

  # Try to open log
  openlog()

  if not os.path.exists('/var/qmail/qmfilt'):
    # Indicate a problem with the queue directory
    log("Directory /var/qmail/qmfilt doesn't exist")
    if not TestMode:
      sys.exit(61)

# #Python 1.5.2 feature
#  if not os.access('/var/qmail/qmfilt', os.W_OK):
#    # Indicate a problem with the queue directory
#    log("Directory /var/qmail/qmfilt doesn't have write permission")
#    if not TestMode:
#      sys.exit(61)

  if os.path.exists('/var/qmail/control/qmfilt'):
    readFilters()
  else:
    if TestMode:
      print "No filter file /var/qmail/control/qmfilt - no filters applied"

  if TestMode:
    showFilters()

  # Go no further if in test mode
  if TestMode:
    sys.exit(0)



  # Get the data

  # Read the data
  MailData     = readDescriptor(0)
  MailLen = len(MailData)
  # Read the envelope 
  MailEnvelope = readDescriptor(1)
  if (MailLen==0):
    sys.exit(0)
  if (MailLen>MaxData or MinData>MailLen):	  
	       log(MailEnvelope+"-GO-size:%d" % MailLen)
	       sendToQmailQueue()
               sys.exit(0)
    	  
  if Debug==9:	  
    log(MailData)
  log(MailEnvelope+"-size:%d" % MailLen)

  action,hit = filterHits()

  if action == 'trap':
    storeInTrap(hit)
  if action == 'block':
    log("Matched filter [ %s] and email was BLOCKED/Refused delivery" %(hit))
    sys.exit(31)
  if action == 'back':
    sendToQmailQueue()
    
    storeInTrap(hit)
  if action == 'drop':
    log("Matched filter [ %s] and email was DROPPED" %(hit))
  if action == None:
    sendToQmailQueue()

  if Debug==9:
    log("qmailqueue returned [%d]" %(result))
  sys.exit(0)

if __name__ == "__main__":
  try:
    main(sys.argv, sys.stdout, os.environ)

  # Catch the sys.exit() errors
  except SystemExit, val:
    sys.exit(val)
  except:
    # return a fatal error for the unknown error
    if TestMode:
      traceback.print_exc()
    sys.exit(81)
   
责任编辑: admin