Monthly Archives: 一月 2012

msn和gtalk机器人

打造自己的gtalk机器人

http://www.venkysblog.com/tutorial-write-your-own-gtalk-custom-bot-in-1

上边的链接中一篇很好快速入门的文章

1. 依赖包：xmpppy和pydns

http://xmpppy.sourceforge.net/

http://pydns.sourceforge.net/

下载完以后。安装

python setup.py install

2. 下载安装gtalkrobot

svn checkout http://pygtalkrobot.googlecode.com/svn/trunk/pygtalkrobot/src pygtalkrobot-read-only

cd pygtalkrobot-read-only

修改sampleRobot.py

最后两行：修改用户名/密码

python sampleRobot.py运行

我加了一段daemon的代码。防止程序死掉

if __name__ == “__main__”:
    try:
        pid = os.fork()
        if pid > 0:
            sys.exit(0)
    except Exception, ex:
        print(“fork1 failn”)
        sys.exit(1)

    os.chdir(‘/’)
    os.setsid()
    os.umask(0)

    while True:
        try:
            pid = os.fork()
            if pid > 0:
                os.wait()
            else:
                break
        except Exception, ex:
            print(“fork2 failn”)
            sys.exit(1)
    bot = SampleBot()
    bot.setState(‘available’, “lava weibo robot (/help查看帮助)”)
    bot.start(“abc@gmail.com”, “abc”)

再有该程序默认的方式为使用message回复。而不是chat的方式。。需要修改PygtalkRobot.py文件中replyMessage函数为

def replyMessage(self, user, message, typ=’chat’):

 

3. 打造自己的机器人.

该项目使用特定函数名称和__doc__保存正则的方式来运行

control代码：

    def initCommands(self):
        if self.commands:
            self.commands.clear()
        else:
            self.commands = list()
        for (name, value) in inspect.getmembers(self):
            if inspect.ismethod(value) and name.startswith(self.command_prefix):
                self.commands.append((value.__doc__, value))
        #print self.commands

    def controller(self, conn, message):
        text = message.getBody()
        user = message.getFrom()
        if text:
            text = text.encode(‘utf-8′, ‘ignore’)
            if not self.commands:
                self.initCommands()
            for (pattern, bounded_method) in self.commands:
                match_obj = re.match(pattern, text)
                if(match_obj):
                    try:
                        return_value = bounded_method(user, text, match_obj.groups())
                        if return_value == self.GO_TO_NEXT_COMMAND:
                            pass
                        else:
                            break
                    except:
                        print_info(sys.exc_info())
                        self.replyMessage(user, traceback.format_exc())
在sampleRobot.py中定义自己要匹配的内容处理函数：

def command_005_addWeibo(self, user, message, args):

    ”’/a weibo (.*)”’

    some code

好了。。到此结束。

 

msn机器人：

http://code.google.com/p/ichatbot/

安装步骤：

  1. Check out latest repository

svn checkout http://ichatbot.googlecode.com/svn/trunk/ ichatbot-read-only

  1. Open chatbot.py, change chatbot MSN email, password, botname and admin_email
  2. Change your bot profile in chatbot.ini such as chatbot’s name, master, birthplace etc
  3. run ./chatbot.py and your robot is ready for chat on MSN
  4. say ‘help’ to your bot on MSN to list available commands

我直接没有使用作者写的程序，自己动手写了一个跟gtalk机器人控制方式一样的程序，这里下载

http://t.lava.cn/blog/26496

 

 

 

 


打造nginx为高效的上传附件服务器

好久没有发文章了。近来一直在研究nginx的upload模块。结合agentzh的nginx-lua模块。放弃后端的php处理。直接使用nginx来做附件服务器。

存在的问题：

1. upload模块的变量upload_file_name变量的生命周期为正在处理中的每一个部分

These variables are valid only during processing of one part of original request body.

2. upload模块的变量upload_file_md5变量的生命周期为上传文件之后

The value of a field specified by this directive is evaluated after successful upload of the file, thus these variables are valid only at the end of processing of one part of original request body.

由于这两个变量的问题。。

可行的解决办法：

1. 刚开始想从body里用正则匹配出file_name和md5。。可是。。对于lua一窍不通的我。。尝试了很多次之后。放弃这条路。。

2. 修改nginx_upload模块。

 

第一条路走不通。硬着头皮走第二条路。。。本人也是刚接触nginx模块的编写。。修改模块还是有一定难度的。。硬着头皮。花了Ｎ天看懂了模块的大致流程。。

1. upload_file_name

upload_discard_part_attributes这个函数里

    upload_ctx->file_name.len = 0;
    upload_ctx->file_name.data = NULL;
把变量销毁。。我靠。。果断注释掉。。。耶。。得到文件名喽。。

 

2. upload_file_md5

这个问题扰了好几个周末。。

一直得到不到正确的md5值。打印出来的时候第一遍是正确的。第二遍就是错误的。在这个变量的get_handler里

static ngx_int_t /* {{{ ngx_http_upload_md5_variable */
ngx_http_upload_md5_variable(ngx_http_request_t *r,
    ngx_http_variable_value_t *v,  uintptr_t data)
{
    ngx_uint_t             i;
    ngx_http_upload_ctx_t  *u;
    u_char                 *c;
    u_char                 *hex_table;

    u = ngx_http_get_module_ctx(r, ngx_http_upload_module);

    if(u->md5_ctx == NULL || u->partial_content) {
        v->not_found = 1;
        return NGX_OK;
    }

    v->valid = 1;
    v->no_cacheable = 0;
    v->not_found = 0;

    hex_table = (u_char*)data;
    c = u->md5_ctx->md5_digest + MD5_DIGEST_LENGTH * 2;

    i = MD5_DIGEST_LENGTH;
    do{
        i–;
        *–c = hex_table[u->md5_ctx->md5_digest[i] & 0xf];
        *–c = hex_table[u->md5_ctx->md5_digest[i] >> 4];
    }while(i != 0);

    v->data = u->md5_ctx->md5_digest;
    v->len = MD5_DIGEST_LENGTH * 2;
    return NGX_OK;
} /* }}} */

里面的逻辑看不太懂。。。应该就是这个变量只能使用一次。第二次使用的时候就会再次根据已有的值计算错误。。。

果断给它增加一个变量。只记录最后一个文件的md5

{ ngx_string(“upload_file_md5_last”), NULL, ngx_http_upload_md5_variable_last,
 0,
 NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 },

/* save last file md5*/
static ngx_int_t
ngx_http_upload_md5_variable_last(ngx_http_request_t *r,
    ngx_http_variable_value_t *v, uintptr_t data)
{
    ngx_http_upload_ctx_t  *u;
    u = ngx_http_get_module_ctx(r, ngx_http_upload_module);

    v->valid = 1;
    v->no_cacheable = 0;
    v->not_found = 0;

    v->data = u->md5_ctx->md5_digest;
    v->len = MD5_DIGEST_LENGTH * 2;
    return NGX_OK;
}

耶。。这个问题也解决了。。。

然后就形如以下nginx配置：

location = /upload {
    upload_pass @test;
    upload_store /data/upload/tmp_upload 1;
    upload_state_store /data/upload/tmp_state 1;
    upload_store_access user:r;
    upload_tame_arrays on;
    upload_pass_args on;

    upload_set_form_field name “$upload_file_name”;
    upload_set_form_field type “$upload_content_type”;
    upload_set_form_field tmp_name “$upload_tmp_path”;
    upload_set_form_field field_name “$upload_field_name”;

    upload_aggregate_form_field md5 “$upload_file_md5″;
    upload_aggregate_form_field size “$upload_file_size”;


    #upload_pass_form_field “$upload_field_name”;
    #upload_pass_form_field “submit”;

    #limit rate
    upload_limit_rate 100k;

    #max file size
    upload_max_file_size 100m;
    #cleanup
    upload_cleanup  400-599;
}

location @test {
    lua_code_cache off;

    set $file_name $upload_file_name;
    set $file_path $upload_tmp_path;
    set $md5 $upload_file_md5_last;
    set $size $upload_file_size;


    content_by_lua_file ‘/data/lua_module/file_upload.lua';
    add_header Content-Type text/html;
}

然后file_upload.lua文件如下：

function isFileExist(md5)
    res = ngx.location.capture(‘/get’,{args={md5 = ‘Attachement::’ .. md5 }})
    return res.status
end

function getRandFileName(ext)
    if ext then
        if not ngx.re.match(ext, “.”, “a”) then
            ext = ‘.’ .. ext
        end
    else
        ext = ”
    end
    math.randomseed(os.date(‘%s’))
    math.random(0,99);math.random(0,99);math.random(0,99);
    return string.char(math.random(97,122)) .. math.random(10000,99999) .. os.date(‘%s’) .. ext
end
function getRandFilePath(n)
    n = tonumber(n) or 2
    local i=0
    local path = ‘/data/upload_file/download/’
    math.randomseed(os.date(‘%s’))
    math.random(0,99);math.random(0,99);math.random(0,99);
    while i < n do
       path = path .. math.random(0,99) .. ‘/’
       i = i + 1
    end
    local cmd = ‘if [ ! -d ‘ .. path .. ‘ ];then mkdir -p ‘ .. path .. ‘ ;fi’
    local ret = os.execute(cmd)
    if ret then
        –ngx.say(‘execute fail’)
    end
    return path
end

size=tonumber(ngx.var.size)
if size > (1024 * 1024 * 100) then
    ngx.exit(413)
end
local res = isFileExist(ngx.var.md5)
if res == 200 then
   ngx.say(‘已有的上传文件!’)
end
local ext = ngx.re.match(ngx.var.file_name, “.([a-z0-9A-Z_-]+)$”, ‘x’)
if ext then
    ext = ext[0]
else
    ext = ”
end
local dest_file_path = getRandFilePath() .. getRandFileName(ext)
local ret = ”
if dest_file_path then
    –ngx.say(dest_file_path)
    ret = os.rename(ngx.var.file_path, dest_file_path)
    if ret then
        –ngx.say(‘上传成功！’)
    end
end
local targetname,n = ngx.re.gsub(dest_file_path, “/data/upload_file/download”, “”)
ngx.say(targetname)

 

好了。再结合ngx_memc和ngx_drizzle后端使用分布式存储就可以抛弃php。nginx自己独自来处理上传业务。。速度绝对soso的。。

感谢agentzh对nginx社区的贡献。。。

修改之后的ngx_upload下载地址：http://t.lava.cn/w/23651