OpenResty ngx 模块介绍
介绍
-iresty/nginx-lua-module-zh-wiki openresty/lua-nginx-module 中文翻译
- 常见的方法
ngx.print("xxx")/ngx.say("xxx")向客户端输出,ngx.say()会自动添加一个换行符ngx.flush(true)可以强制刷新缓冲区,同步非阻塞
ngx.get_phase()获取当前处理阶段ngx.config.*包含大量信息ngx.encode_args(args)ngx.decode_args(enc)- 时间
ngx.update_time()强制更新内部缓存的时间,会调用系统函数gettimeofday(),资源消耗高ngx.today()yyyy-mm-ddngx.localtime()yyyy-mm-dd hh:mm:ssngx.utctime()yyyy-mm-dd hh:mm:ssngx.time当前时间戳,秒ngx.now当前时间戳,毫秒ngx.http_time()将时间戳转化为 http 时间格式ngx.cookie_time()将时间戳转化为 cookie 时间格式ngx.parse_http_time()将 http 时间转化为时间戳ngx.sleep()同步非阻塞睡眠函数,不能使用在init_by_lua/init_worker_by_lua/set_by_lua/header/body_filter_by_lua/log_by_lua等阶段
- 流程控制
ngx.redirect(url, status)重定向ngx.exec(uri, args)跳转到内部其他 locationngx.exit(status)立即结束请求ngx.eof()发送 EOF 标志,后续不再有响应数据ngx.process进程管理ngx.workerworker 管理
ngx.encode_base64("xxx"[, true])/ngx.decode_base64("xxx")还有其他函数如encode_base64url/decode_base64urlngx.worker.id()
HTTP method
https://github.com/openresty/lua-nginx-module#http-method-constants
- ngx.HTTP_GET
- ngx.HTTP_HEAD
- ngx.HTTP_PUT
- ngx.HTTP_POST
- ngx.HTTP_DELETE
- ngx.HTTP_OPTIONS (added in the v0.5.0rc24 release)
- ngx.HTTP_MKCOL (added in the v0.8.2 release)
- ngx.HTTP_COPY (added in the v0.8.2 release)
- ngx.HTTP_MOVE (added in the v0.8.2 release)
- ngx.HTTP_PROPFIND (added in the v0.8.2 release)
- ngx.HTTP_PROPPATCH (added in the v0.8.2 release)
- ngx.HTTP_LOCK (added in the v0.8.2 release)
- ngx.HTTP_UNLOCK (added in the v0.8.2 release)
- ngx.HTTP_PATCH (added in the v0.8.2 release)
- ngx.HTTP_TRACE (added in the v0.8.2 release)
HTTP status
https://github.com/openresty/lua-nginx-module#http-status-constants
value = ngx.HTTP_OK (200)
value = ngx.HTTP_CREATED (201)
value = ngx.HTTP_ACCEPTED (202)
value = ngx.HTTP_BAD_REQUEST (400)
value = ngx.HTTP_UNAUTHORIZED (401)
value = ngx.HTTP_PAYMENT_REQUIRED (402) (first added in the v0.9.20 release)
value = ngx.HTTP_FORBIDDEN (403)
value = ngx.HTTP_NOT_FOUND (404)
value = ngx.HTTP_NOT_ALLOWED (405)
...ngx.ctx
- 每个请求,包括子请求,都有一份自己的
ngx.ctx表,可以在不同阶段共享变量
示例
location /sub {
content_by_lua_block {
ngx.say("sub pre: ", ngx.ctx.blah)
ngx.ctx.blah = 32
ngx.say("sub post: ", ngx.ctx.blah)
}
}
location /main {
content_by_lua_block {
ngx.ctx.blah = 73
ngx.say("main pre: ", ngx.ctx.blah)
local res = ngx.location.capture("/sub")
ngx.print(res.body)
ngx.say("main post: ", ngx.ctx.blah)
}
}输出:
main pre: 73
sub pre: nil
sub post: 32
main post: 73ndk.set_var方式 SQL 注入,如ndk.set_var.set_quote_sql_str(req_id))
ngx.header
ngx.header.content_length = 0ngx.header['Server'] = 'x'ngx.header.date = nil删除数据
ngx.location
子请求:
ngx.location.capture(uri, optiions)仅用在 rewrite/access/content_by_lua 3 个阶段- 参数说明
uriNGINX 的 locationoptiions表包括的字段method请求方法argsurl 参数表body数据ctxngx.ctx 临时数据vars变量表
- 返回值 status, header, body, truncated
status状态码,相当于 ngx.statusheader响应体,相当于 ngx.headerbody响应体truncated错误标识,body 数据是否被意外截断
- 参数说明
ngx.location.capture_multi({ {uri, optiions}, {uri, optiions}, {uri, optiions}, ... })同时发起多个子请求,最多只能发起 50 个子请求
local capture = ngx.location.capture
local res = capture(uri,
{method = ngx.HTTP_POST,
args = { ... },
body = "xxx"
})
if res.status = ngx.HTTP_OK then
ngx.print(res.body)
end
if res.status ~= ngx.HTTP_OK then
ngx.exit(res.status)
end
if res.truncated then
ngx.log(ngx.ERR, "xxx")
endngx.log
https://github.com/openresty/lua-nginx-module#nginx-log-level-constants
- 标准日志输出
ngx.log(log_level, ...)ngx.log(ngx.STDERR, ...)标准输出ngx.log(ngx.EMERG, ...)紧急输出ngx.log(ngx.ALERT, ...)告警错误ngx.log(ngx.CRIT, ...)严重错误ngx.log(ngx.ERR, "error:", errstr)错误信息ngx.log(ngx.NOTICE, ...)提醒信息ngx.log(ngx.INFO, "info:", infostr)一般信息ngx.log(ngx.DEBUG, "debug:", debugstr)调试信息
- OpenResty Lua 调试一般通过打印日志实现
location /test-log {
content_by_lua_block {
local i = "xxxx";
ngx.log(ngx.ERR, "some error...", i);
ngx.log(ngx.INFO, "some info...", i);
ngx.log(ngx.DEBUG, "some debug...", i);
ngx.say("log done");
}
}调用:
$ curl '127.0.0.1/test-log'
log done
# error.log 日志
2022/09/11 09:25:29 [error] 2887#0: *13 [lua] content_by_lua(default.conf:57):3: some error...xxxx, client: 127.0.0.1, server: localhost, request: "GET /test-log HTTP/1.1", host: "127.0.0.1"ngx.md5
ngx.md5("strs")ngx.re
正则表达式处理库,尽量将字符串放到 [[...]] 中,防止转移
match单次匹配gmatch多次匹配find查找,返回索引位置sub正则替换gsub多次正则替换split正则切分
OpenResty 中优化正则表达式
lua_regex_cache_max_entries num默认为 1024lua_regex_match_limit num正则表达式匹配时回溯的最大次数,小值可以提高运行效率
ngx.req
ngx.req.is_internal()是否内部请求local req_time = ngx.now() - nex.req.start_time()ngx.req.get_method()ngx.req.set_method(ngx.HTTP_GET)ngx.req.set_uri(uri, jump)ngx.req.set_body_data()设置请求体数据ngx.req.init_body()新建请求体ngx.req.append_body("xxx")添加数据ngx.req.finish_body()完成请求体创建ngx.req.read_body读取数据/lua_need_request_body onlocal sock, err = ngx.req.socket(); local data = sock:receive(lan)
ngx.req.read_body ()
local data = ngx.req.get_body_data()
if data then
ngx.say('body: ', data)
else
local name = ngx.req.get_body_file()
local f = io.open(name, 'r')
data = f:read('*a')
f:close()
end获取请求 header
ngx.req.raw_header()原始字符串ngx.req.get_headers()- 通过
.获取的结果变化有两点:完全小写化;-转换为_ - 通过
[]可以使用原始形式获取,headers['Accept']
- 通过
ngx.req.set_headers(key, value)- 删除
ngx.req.set_headers('key', nil)或ngx.req.clean_header('key')
- 删除
ngx.req.discard_body()丢弃请求体
获取 uri 参数
获取 uri 参数有两个方法:
ngx.req.get_uri_args(max_args)ngx.req.set_uri_args(args)ngx.req.get_post_args(max_args)
location /print_param {
content_by_lua_block {
local arg = ngx.req.get_uri_args()
for k,v in pairs(arg) do
ngx.say("[GET ] key:", k, " v:", v)
end
ngx.req.read_body() -- 解析 body 参数之前一定要先读取 body
local arg = ngx.req.get_post_args()
for k,v in pairs(arg) do
ngx.say("[POST] key:", k, " v:", v)
end
}
}
location /test-print-param {
content_by_lua_block {
local res = ngx.location.capture(
'/print_param',
{
method = ngx.HTTP_POST,
args = ngx.encode_args({a = 1, b = '2&'}),
body = ngx.encode_args({c = 3, d = '4&'})
}
)
ngx.say(res.body)
}
}- 调用
# %26 是 & 符
$ curl '127.0.0.1/print_param?a=1&b=2%26' -d 'c=3&d=4%26'
[GET ] key:b v:2&
[GET ] key:a v:1
[POST] key:d v:4&
[POST] key:c v:3
# url 参数传递
$ curl '127.0.0.1/test-print-param'
[GET ] key:a v:1
[GET ] key:b v:2&
[POST] key:c v:3
[POST] key:d v:4&- 其他修改方式
location /proxy {
internal;
rewrite_by_lua_block {
-- 获取原始请求的查询参数
local args = ngx.req.get_uri_args()
-- 添加新参数
args["from"] = "nginx"
-- 设置新的查询参数
ngx.req.set_uri_args(args)
-- 转发到目标地址
ngx.exec("@target")
}
}
location @target {
proxy_pass http://backend_server;
}
# 或
location /api {
rewrite_by_lua_block {
-- 获取当前URI参数
local args = ngx.req.get_uri_args()
-- 添加新参数
args.from = "nginx"
-- 重新设置URI参数
ngx.req.set_uri_args(args)
}
proxy_pass http://backend_server;
}
# 或
location /forward {
content_by_lua_block {
local http = require "resty.http"
local httpc = http.new()
-- 获取原始参数
local args = ngx.req.get_uri_args()
-- 添加新参数
args.from = "nginx"
-- 转发请求
local res, err = httpc:request_uri("http://backend_server/api", {
method = ngx.req.get_method(),
args = args,
headers = ngx.req.get_headers()
})
if not res then
ngx.log(ngx.ERR, "request failed: ", err)
ngx.exit(500)
end
-- 返回响应
ngx.status = res.status
for k, v in pairs(res.headers) do
ngx.header[k] = v
end
ngx.print(res.body)
}
}获取请求 body
http {
server {
listen 80;
# ngx.req.read_body() 和 lua_need_request_body 必须配置一个,否则读不到数据
# lua_need_request_body on;
location /test-get-body-data {
content_by_lua_block {
ngx.req.read_body()
local data = ngx.req.get_body_data()
ngx.say("hello ", data)
}
}
}
}说明:
ngx.req.read_body()和lua_need_request_body: on;必须配置一个,否则读不到数据- 在
OpenResty中,HTTP 响应体的输出可调用(均为异步输出)ngx.say(多输出一个\n)ngx.print
- 显式的向客户端刷新响应输出
ngx.flush()
调用:
$ curl '127.0.0.1/test-get-body-data' -d 'world'
hello worldngx.resp
-
ngx.resp.add_header("key", "value")添加 header -
ngx.send_headers()发送响应头 -
ngx.headers_send是否发送相应头标志 -
示例 1
ngx.resp.get_headers(0, true)
-- 从 header 获取 ip
local headers=ngx.req.get_headers()
local ip = headers["X-REAL-IP"] or headers["X_FORWARDED_FOR"] or ngx.var.remote_addr or "0.0.0.0"
# 自定义 header
ngx.header.Foo = 'Bar'
-- Foo: Bar
-- _ 会自动转化为 -
ngx.header['Foo_Bar'] = {'a=123; path=/', 'b=4; path=/'}
-- Foo-Bar: a=123; path=/
-- Foo-Bar: b=4; path=/
# 删除 header
ngx.header["X-Header"] = nil;
ngx.header["X-Header"] = {};- 示例 2
-- curl -H "Foo: bar" 127.0.0.1:80/api/test_header
local function print_table(t)
local function parse_array(key, tab)
local str = ''
for _, v in pairs(tab) do
str = str .. key .. ' ' .. v .. '\r\n'
end
return str
end
local str = ''
for k, v in pairs(t) do
if type(v) == "table" then
str = str .. parse_array(k, v)
else
str = str .. k .. ' ' .. (v) .. '\r\n'
end
end
return str
end
-- local headers = ngx.req.get_headers() -- 请求 Headers
local headers = ngx.resp.get_headers(0, true) -- 响应 Headers
ngx.say(print_table(headers))ngx.semaphore
- 使用信号量 semaphore 实现 OpenResty 在本进程的不同线程之间同步功能
- 不同进程的同步使用共享内存实现
local semaphore = require "ngx.semaphore"
-- new 新建;wait 等待信号量;post 新增信号量;count 获取信号量数量
-- 初始资源数量,默认为 0
local sema = semaphore.new(0)
sema:post()
sema:wait(1) -- 等待1s获取信号量
-- 配合共享内存实现数据传递ngx.shared.shmem
定义共享内存
loca l shmem = ngx.shared.shmem
local ok, err, f
-- 设置
ok, err, f = shmem:set("num", 1, 0.05) -- 0.05 秒后过期
assert(ok and not f)
ok, err = shmem:add("num", 1)
assert (not ok)
ok, err = shmem:replace("num", 2)
assert (ok)
ok, err = shmem:add("ver", 1, 0, 1)
assert (ok)
-- 获取
local v , flags = shmem:get ("ver")
assert(v == 1 and flags == 1)
ngx.sleep(0.2)
local v, err, stale = shmem:get_stale("num")
-- 删除
shmem:delete("num")
-- 计数
local v = shmem:incr("count", 1, 0)
local v = shmem:incr("count", 5)
-- 队列操作
local len = shmem:lpush("list", "a")
local len = shmem:rpush("list", "z")
local len = shmem:llen("list")
local v = shmem:lpop("list")
local v = shmem:rpop("list")ngx.socket
-
ngx.socket.tcp -
ngx.socket.udp -
cosocket=coroutine base socket以同步非阻塞方式实现,效率高,并支持连接池机制- coroutine:协程
- socket:网络套接字
配置参数:
-
lua_socket_log_errors on|off错误日志 -
lua_socket_send_lowat num缓存,发送数据的阈值(low water),超过才发送,默认为 0,立即发送 -
lua_socket_buffer_size num接收数据的缓冲区大小,默认值是4KB/8KB -
lua_socket_connect_timeout time连接后端的超时时间,默认 60s -
lua_socket_send_timeout time发送数据的超时时间,默认 60s -
lua_socket_read_timeout time接收数据的超时时间,默认 60s -
lua_socket_pool_size numcosocket 连接池的大小,默认值是 30,可以大些提高效率 -
lua_socket_keepalive_timeout time连接池里 cosocket 对象的空闲时间,默认 60s -
test_socket.lua
local sock = ngx.socket.tcp()
-- sock:settimeout(1000) -- 超时时间 1000 毫秒
-- sock:settimeouts(connect_time, send_timeout, read_time)
local ok, err = sock:connect("www.baidu.com", 80)
if not ok then
ngx.say("failed to connect to baidu: ", err)
return
end
-- 放入连接池,timeout 指定连接空闲时间,单位是毫秒;size 连接池大小
-- ok, err = sock:setkeepalive(timeout, size)
-- 获取复用次数
-- count, err = sock:getreusedtimes()
local req_data = "GET / HTTP/1.1\r\nHost: www.baidu.com\r\n\r\n"
-- 同步非阻塞发送数据
local bytes, err = sock:send(req_data)
if err then
ngx.say("failed to send to baidu: ", err)
return
end
-- 同步非阻塞接收数据,*a 表示接受所有数据
local data, err, partial = sock:receive("*a")
if err then
ngx.say("failed to receive from baidu: ", err)
return
end
-- 关闭连接
sock:close()
ngx.say("successfully talk to baidu! response first line: ", data)ngx.thread
OpenResty 线程介绍
local spawn = ngx.thread.spawn
local wait = ngx.thread.wait
local function sayhi(world)
ngx.say("hi ", world)
end
-- 启动线程
-- t = spawn(func, arg1, arg2, ...)
local threads = {
spawn(sayhi, "world"),
spawn(sayhi, "world")
spawn(sayhi, "world")
}
-- 等待线程
-- ok, ... = ngx.thread.wait(task1, task2, ...)
local ok, v = wait(unpack(threads))
for i, t in ipairs(threads) do
local ok, v = wait(t)
ngx.sya("wait ", v)
end
-- 获取所有协程对象
co = coroutine.running()
-- 挂起线程
coroutine.yield(co)
-- 停止线程
local kill = ngx.thread.kill
kill(t)ngx.timer 定时任务
OpenResty 后台处理某些作业(数据定期清理、同步数据),期望只有一个实例运行,可以通过 ngx.worker.id() 获取 worker_id,让特定的 worker_id 运行定时任务
- 若 worker 是 n,那么
ngx.worker.id()返回0 ~ n-1的一个数字
init_worker_by_lua_block {
local delay = 3 -- in seconds
local timer_at = ngx.timer.at
local log = ngx.log
local ERR = ngx.ERR
local task = function(premature, params)
if premature then
return
end
log(ERR, "run ...")
new_timer(delay, task) -- maybe need check result
end
if 1 == ngx.worker.id() then
local ok, err = new_timer(delay, task)
if not ok then
log(ERR, "failed to create timer: ", err)
return
end
end
}ngx.upstream
local names = upstream.get_upstreams()
for i, n = in ipairs(names) do
local srvs , err = upstream.get_servers(n)
if not srvs then
ngx.say("failed to get servers in ", n)
goto continue
end
ngx.say("upstream : ", n)
for i,s in ipairs(srvs) do
for k,v in pairs(s) do
ngx.print(k, "=", v, ";")
end
end
::continue::
end
# 下线
upstream.set_peer_down(...)- 异步非阻塞的
- 动态 upstream:OpenResty 的
balancer_by_lua指令让动态负载均衡称为可能,它替代了原生的hash/ip_hash/least_conn等算法,不仅可以让自由定制负载均衡策略,还可以随意调整后端服务器的数量
upstream dyn backend { #动态上游集群
server 0.0.0.0; # 无实际意义的占位用
balancer_by_lua_file proxy/balancer.lua;
keepalive 10;
}
$ cat proxy/balancer.lua
local balancer = require "ngx.balancer"
local servers = {
{"127.0.0.1", 80},
{"127.0.0.1", 8080},
}
balancer.set_timeouts(1, 0.5, 0.5)
balancer.set_more_tries(2)
local n = math.random(#servers)
local ok, err = balancer.set_current_peer(servers[n][1], servers[n][2])
if not ok then
ngx.log(ngx.ERR, "failed to set peer:", err)
return ngx.exit(500)
end- 建议使用
lua-resty-balancer库
ngx.var
ngx.var 包含内置变量和自定义变量,可以通过 . 或 [] 访问
ngx.var.uri对应$uringx.var['request_length']对应$request_lengthngx.var.request_uringx.var.server_name和ngx.var.server_portngx.var.remote_addr和ngx.var.remote_port
location /foo {
set $my_var ''; # this line is required to create $my_var at config time
content_by_lua_block {
ngx.var.my_var = 123
...
// 删除变量
ngx.var.my_var = nil
}
}- 从环境获取值
init_by_lua 'key = os.getenv("KEY")'
...
server {
set_by_lua $key 'return key';
location /get {
echo $key;
}
}ngx.on_abort
客户端断开链接时,可以使用 ngx.on_abort 注册函数,处理资源回收等
local function cleanup()
ngx.log(ngx.ERR, "something clean...")
ngx.exit(400)
end
local ok, err = ngx.on_abort(cleanup)状态码
- ngx.HTTP_OK 200
- ngx.HTTP_ MOVED_TEMPORARILY 302
- ngx.HTTP_ BAD 400
- ngx.HTTP UNAUTHORIZED 401
- ngx.HTTP_NOT_FOUND 404
- ngx.HTTP_INTERNAL_SERVER_ERROR 500
- ngx.HTTP_BAD_GATEWAY 502
- ngx.HTTP_SERVICE_UNAVAILABLE 503
- ngx.HTTP_GATEWAY_TIMEOUT 504
请求方法
- ngx.HTTP_GET
- ngx.HTTP_HEAD
- ngx.HTTP_POST
- ngx.HTTP_PUT
- ngx.HTTP_DELETE
- ngx.HTTP_PATCH