资讯专栏INFORMATION COLUMN

Bottle源码阅读(一)

whidy / 1116人阅读

摘要:在初识一中,我们了解了框架的基本用法。在本篇文章中,我们通过源码来探究一些基本原理。因此下一步就是研究我们写的应用函数是如何被封装成适配的

</>复制代码

  1. 在初识bottle(一)中,我们了解了bottle框架的基本用法。在本篇文章中,我们通过源码来探究一些基本原理。
1. run的实现

所有的框架请求响应都基于一个原理
http请求 --> wsgi服务器 --> wsgi接口(实际就是框架中自定义实现的函数经过底层封装) --> 响应
可以参考廖雪峰的教程中关于wsgi接口的讲解

下我们先看看bottle是如何实现服务器运行时自动重新加载

</>复制代码

  1. def run(app=None,
  2. server="wsgiref",
  3. host="127.0.0.1",
  4. port=8080,
  5. interval=1,
  6. reloader=False,
  7. quiet=False,
  8. plugins=None,
  9. debug=None,
  10. config=None, **kargs):
  11. """ Start a server instance. This method blocks until the server terminates.
  12. :param app: WSGI application or target string supported by
  13. :func:`load_app`. (default: :func:`default_app`)
  14. :param server: Server adapter to use. See :data:`server_names` keys
  15. for valid names or pass a :class:`ServerAdapter` subclass.
  16. (default: `wsgiref`)
  17. :param host: Server address to bind to. Pass ``0.0.0.0`` to listens on
  18. all interfaces including the external one. (default: 127.0.0.1)
  19. :param port: Server port to bind to. Values below 1024 require root
  20. privileges. (default: 8080)
  21. :param reloader: Start auto-reloading server? (default: False)
  22. :param interval: Auto-reloader interval in seconds (default: 1)
  23. :param quiet: Suppress output to stdout and stderr? (default: False)
  24. :param options: Options passed to the server adapter.
  25. """
  26. if NORUN: return
  27. # 自动重载
  28. if reloader and not os.environ.get("BOTTLE_CHILD"):
  29. import subprocess
  30. lockfile = None
  31. try:
  32. # tempfile 临时文件操作模块https://docs.python.org/2/library/tempfile.html
  33. # 第一个相当于执行os.open()函数返回文件handler,第二个表示绝对路径
  34. fd, lockfile = tempfile.mkstemp(prefix="bottle.", suffix=".lock")
  35. os.close(fd) # We only need this file to exist. We never write to it
  36. # sys.executable 是获取当前python解释器的路径
  37. while os.path.exists(lockfile):
  38. args = [sys.executable] + sys.argv
  39. environ = os.environ.copy()
  40. environ["BOTTLE_CHILD"] = "true"
  41. environ["BOTTLE_LOCKFILE"] = lockfile
  42. # 创建一个子进程实例
  43. p = subprocess.Popen(args, env=environ)
  44. # 如果返回None表示子进程未结束
  45. while p.poll() is None: # Busy wait...
  46. # 临时文件设置为当前时间
  47. os.utime(lockfile, None) # I am alive!
  48. time.sleep(interval)
  49. # linux 系统的信号机制http://www.cppblog.com/sleepwom/archive/2010/12/27/137564.html
  50. # 3表示按下退出键
  51. # 非正常退出时
  52. if p.poll() != 3:
  53. # os.unlink 相当于去除remove()
  54. if os.path.exists(lockfile): os.unlink(lockfile)
  55. sys.exit(p.poll())
  56. except KeyboardInterrupt:
  57. pass
  58. finally:
  59. if os.path.exists(lockfile):
  60. os.unlink(lockfile)
  61. return

首先第一次运行时,开启一个新的进程,确保运行server时的进程和python解释器一致
不影响主进程的继续运行

</>复制代码

  1. try:
  2. # 这一部分主要是app的相关设置
  3. if debug is not None: _debug(debug)
  4. app = app or default_app()
  5. if isinstance(app, basestring):
  6. app = load_app(app)
  7. if not callable(app):
  8. raise ValueError("Application is not callable: %r" % app)
  9. for plugin in plugins or []:
  10. if isinstance(plugin, basestring):
  11. plugin = load(plugin)
  12. app.install(plugin)
  13. if config:
  14. app.config.update(config)
  15. if server in server_names:
  16. server = server_names.get(server)
  17. if isinstance(server, basestring):
  18. server = load(server)
  19. if isinstance(server, type):
  20. server = server(host=host, port=port, **kargs)
  21. if not isinstance(server, ServerAdapter):
  22. raise ValueError("Unknown or unsupported server: %r" % server)
  23. server.quiet = server.quiet or quiet
  24. if not server.quiet:
  25. _stderr("Bottle v%s server starting up (using %s)...
  26. " %
  27. (__version__, repr(server)))
  28. _stderr("Listening on http://%s:%d/
  29. " %
  30. (server.host, server.port))
  31. _stderr("Hit Ctrl-C to quit.
  32. ")
  33. # 当选择自动重载时,如果解释器进程已经启动
  34. # 则只需要检测应用相关内容有没有变化,如果有变化终止主线程并重新实现异常捕获
  35. if reloader:
  36. lockfile = os.environ.get("BOTTLE_LOCKFILE")
  37. bgcheck = FileCheckerThread(lockfile, interval)
  38. # 开启新线程检测文件修改,如果修改终止当前主线程,抛出异常
  39. with bgcheck:
  40. # 主线程监听请求
  41. server.run(app)
  42. if bgcheck.status == "reload":
  43. sys.exit(3)
  44. else:
  45. server.run(app)
  46. except KeyboardInterrupt:
  47. pass
  48. except (SystemExit, MemoryError):
  49. raise
  50. except:
  51. if not reloader: raise
  52. if not getattr(server, "quiet", quiet):
  53. print_exc()
  54. time.sleep(interval)
  55. sys.exit(3)

FileCheckerThread会对应用相关文件内容变化进行检测
server加载app,由server接收请求并执行相应的应用函数
在此之前,我们先了解FileCheckerThread

2. 应用修改后的自动重载

这是一个上下文管理器,当__enter__时开启一个新的线程,这个线程的任务就是检测应用相关模块文件的变化,决定是否终止主线程,当__exit__时,如果返回True则重现异常,否则正常执行后续代码

</>复制代码

  1. class FileCheckerThread(threading.Thread):
  2. """ Interrupt main-thread as soon as a changed module file is detected,
  3. the lockfile gets deleted or gets too old. """
  4. def __init__(self, lockfile, interval):
  5. threading.Thread.__init__(self)
  6. self.daemon = True
  7. self.lockfile, self.interval = lockfile, interval
  8. #: Is one of "reload", "error" or "exit"
  9. self.status = None
  10. def run(self):
  11. exists = os.path.exists
  12. mtime = lambda p: os.stat(p).st_mtime
  13. files = dict()
  14. for module in list(sys.modules.values()):
  15. path = getattr(module, "__file__", "")
  16. if path[-4:] in (".pyo", ".pyc"): path = path[:-1]
  17. if path and exists(path): files[path] = mtime(path)
  18. while not self.status:
  19. if not exists(self.lockfile)
  20. or mtime(self.lockfile) < time.time() - self.interval - 5:
  21. self.status = "error"
  22. thread.interrupt_main()
  23. for path, lmtime in list(files.items()):
  24. if not exists(path) or mtime(path) > lmtime:
  25. self.status = "reload"
  26. thread.interrupt_main()
  27. break
  28. time.sleep(self.interval)
  29. def __enter__(self):
  30. self.start()
  31. # 这个地方是重新载入更新后模块的关键
  32. # 当检测到文件变化时,终止主线程使监听请求停止,退出上下文管理器时,如果返回True则重现异常捕获
  33. def __exit__(self, exc_type, *_):
  34. if not self.status: self.status = "exit" # silent exit
  35. self.join()
  36. return exc_type is not None and issubclass(exc_type, KeyboardInterrupt)
3. server调用应用函数

bottle提供了一个ServerAdapter的适配器类,重写run方法就能使bottle可以使用多种框架提供的server。

</>复制代码

  1. class ServerAdapter(object):
  2. quiet = False
  3. def __init__(self, host="127.0.0.1", port=8080, **options):
  4. self.options = options
  5. self.host = host
  6. self.port = int(port)
  7. def run(self, handler): # pragma: no cover
  8. pass
  9. def __repr__(self):
  10. args = ", ".join(["%s=%s" % (k, repr(v))
  11. for k, v in self.options.items()])
  12. return "%s(%s)" % (self.__class__.__name__, args)

默认使用了python自带的wsgiref, 从代码中我们可以看到其中主要由三部分组成:接收请求模块,处理请求模块,组装模块

</>复制代码

  1. class WSGIRefServer(ServerAdapter):
  2. def run(self, app): # pragma: no cover
  3. from wsgiref.simple_server import make_server
  4. from wsgiref.simple_server import WSGIRequestHandler, WSGIServer
  5. import socket
  6. class FixedHandler(WSGIRequestHandler):
  7. def address_string(self): # Prevent reverse DNS lookups please.
  8. return self.client_address[0]
  9. def log_request(*args, **kw):
  10. if not self.quiet:
  11. return WSGIRequestHandler.log_request(*args, **kw)
  12. handler_cls = self.options.get("handler_class", FixedHandler)
  13. server_cls = self.options.get("server_class", WSGIServer)
  14. if ":" in self.host: # Fix wsgiref for IPv6 addresses.
  15. if getattr(server_cls, "address_family") == socket.AF_INET:
  16. class server_cls(server_cls):
  17. address_family = socket.AF_INET6
  18. self.srv = make_server(self.host, self.port, app, server_cls,
  19. handler_cls)
  20. self.port = self.srv.server_port # update port actual port (0 means random)
  21. try:
  22. self.srv.serve_forever()
  23. except KeyboardInterrupt:
  24. self.srv.server_close() # Prevent ResourceWarning: unclosed socket
  25. raise
4.WSGIServer

4.1 寻根到底,我们现研究一下WSGIServer 的基类
BaseServer 主要实现线程上的控制,实现一些供上层调用的接口,例如

</>复制代码

  1. server_activate
  2. serve_forever
  3. shutdown
  4. handle_request
  5. verify_request
  6. handle_error

TCPServer 继承BaseServer, 实现bind,listen,accept, close等函数的封装

</>复制代码

  1. def server_bind(self):
  2. """Called by constructor to bind the socket.
  3. May be overridden.
  4. """
  5. if self.allow_reuse_address:
  6. self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
  7. self.socket.bind(self.server_address)
  8. self.server_address = self.socket.getsockname()
  9. def server_activate(self):
  10. """Called by constructor to activate the server.
  11. May be overridden.
  12. """
  13. self.socket.listen(self.request_queue_size)
  14. def server_close(self):
  15. """Called to clean-up the server.
  16. May be overridden.
  17. """
  18. self.socket.close()

HttpServer 继承TCPServer, 添加了host和port两个属性
WSGIServer 继承HttpServer, 设置了环境变量,提供了获取应用和设置应用的接口

</>复制代码

  1. class WSGIServer(HTTPServer):
  2. """BaseHTTPServer that implements the Python WSGI protocol"""
  3. application = None
  4. def server_bind(self):
  5. """Override server_bind to store the server name."""
  6. HTTPServer.server_bind(self)
  7. self.setup_environ()
  8. def setup_environ(self):
  9. # Set up base environment
  10. env = self.base_environ = {}
  11. env["SERVER_NAME"] = self.server_name
  12. env["GATEWAY_INTERFACE"] = "CGI/1.1"
  13. env["SERVER_PORT"] = str(self.server_port)
  14. env["REMOTE_HOST"]=""
  15. env["CONTENT_LENGTH"]=""
  16. env["SCRIPT_NAME"] = ""
  17. def get_app(self):
  18. return self.application
  19. def set_app(self,application):
  20. self.application = application

4.2 WSGIRequestHandler的实现
最底层的BaseRequestHandler:处理请求的基类,定义了处理请求的流程
StreamRequestHandler: 继承BaseRequestHandler,提供了处理请求前rfile和wfile属性,使处理请求时能通过类似文件读写获取请求和返回响应

</>复制代码

  1. class StreamRequestHandler(BaseRequestHandler):
  2. """Define self.rfile and self.wfile for stream sockets."""
  3. # Default buffer sizes for rfile, wfile.
  4. # We default rfile to buffered because otherwise it could be
  5. # really slow for large data (a getc() call per byte); we make
  6. # wfile unbuffered because (a) often after a write() we want to
  7. # read and we need to flush the line; (b) big writes to unbuffered
  8. # files are typically optimized by stdio even when big reads
  9. # aren"t.
  10. rbufsize = -1
  11. wbufsize = 0
  12. # A timeout to apply to the request socket, if not None.
  13. timeout = None
  14. # Disable nagle algorithm for this socket, if True.
  15. # Use only when wbufsize != 0, to avoid small packets.
  16. disable_nagle_algorithm = False
  17. def setup(self):
  18. self.connection = self.request
  19. if self.timeout is not None:
  20. self.connection.settimeout(self.timeout)
  21. if self.disable_nagle_algorithm:
  22. self.connection.setsockopt(socket.IPPROTO_TCP,
  23. socket.TCP_NODELAY, True)
  24. self.rfile = self.connection.makefile("rb", self.rbufsize)
  25. self.wfile = self.connection.makefile("wb", self.wbufsize)
  26. def finish(self):
  27. if not self.wfile.closed:
  28. try:
  29. self.wfile.flush()
  30. except socket.error:
  31. # A final socket error may have occurred here, such as
  32. # the local error ECONNABORTED.
  33. pass
  34. self.wfile.close()
  35. self.rfile.close()

BaseHTTPRequestHandler:继承StreamRequestHandler,handle处理一个请求,轮询直到收到一个明确关闭连接;parse_request解析请求requestline,如果一切正常,继续处理请求

WSGIRequestHandler:继承了BaseHTTPRequestHandler, 添加get_environ获取环境变量, 重写了handle方法。当requestline >65536时返回414, 实例化一个ServerHandler实例

</>复制代码

  1. def handle(self):
  2. """Handle a single HTTP request"""
  3. self.raw_requestline = self.rfile.readline(65537)
  4. if len(self.raw_requestline) > 65536:
  5. self.requestline = ""
  6. self.request_version = ""
  7. self.command = ""
  8. self.send_error(414)
  9. return
  10. if not self.parse_request(): # An error code has been sent, just exit
  11. return
  12. handler = ServerHandler(
  13. self.rfile, self.wfile, self.get_stderr(), self.get_environ()
  14. )
  15. handler.request_handler = self # backpointer for logging
  16. handler.run(self.server.get_app())

handler.run(self.server.get_app())实现了从请求到应用函数执行,并把执行后的结果写入wfile返回
我们再看wsgiref.handlers中BaseHandler中,是如何实现的。

</>复制代码

  1. def run(self, application):
  2. """Invoke the application"""
  3. # Note to self: don"t move the close()! Asynchronous servers shouldn"t
  4. # call close() from finish_response(), so if you close() anywhere but
  5. # the double-error branch here, you"ll break asynchronous servers by
  6. # prematurely closing. Async servers must return from "run()" without
  7. # closing if there might still be output to iterate over.
  8. try:
  9. self.setup_environ()
  10. self.result = application(self.environ, self.start_response)
  11. self.finish_response()
  12. except:
  13. try:
  14. self.handle_error()
  15. except:
  16. # If we get an error handling an error, just give up already!
  17. self.close()
  18. raise # ...and let the actual server figure it out.
  19. def start_response(self, status, headers,exc_info=None):
  20. """"start_response()" callable as specified by PEP 333"""
  21. if exc_info:
  22. try:
  23. if self.headers_sent:
  24. # Re-raise original exception if headers sent
  25. raise exc_info[0], exc_info[1], exc_info[2]
  26. finally:
  27. exc_info = None # avoid dangling circular ref
  28. elif self.headers is not None:
  29. raise AssertionError("Headers already set!")
  30. assert type(status) is StringType,"Status must be a string"
  31. assert len(status)>=4,"Status must be at least 4 characters"
  32. assert int(status[:3]),"Status message must begin w/3-digit code"
  33. assert status[3]==" ", "Status message must have a space after code"
  34. if __debug__:
  35. for name,val in headers:
  36. assert type(name) is StringType,"Header names must be strings"
  37. assert type(val) is StringType,"Header values must be strings"
  38. assert not is_hop_by_hop(name),"Hop-by-hop headers not allowed"
  39. self.status = status
  40. self.headers = self.headers_class(headers)
  41. return self.write

application接受了两个参数,一个envrion, 和一个start_response的方法。因此下一步就是研究我们写的应用函数是如何被封装成适配的application

文章版权归作者所有,未经允许请勿转载,若此文章存在违规行为,您可以联系管理员删除。

转载请注明本文地址:https://www.ucloud.cn/yun/38449.html

相关文章

  • Bottle源码阅读(二)

    摘要:在源码阅读一中,我们了解了如何接收请求,处理请求以及如何检测模块变化重启。接下来我们看一下源码是怎么实现的经过封装后,最终获得的是具备有一些属性的装饰器当为时,将的属性传递给,使其具备相同的属性。 在《Bottle源码阅读(一)》中,我们了解了bottle如何接收请求,处理请求以及如何检测模块变化重启server。在ServerHandler类中的run函数中,application接...

    zzbo 评论0 收藏0
  • 初识Bottle(二)

    摘要:而其他的引擎,例如能够帮我们进行验证登录自此,官网的我们已经大致有了了解后续我们可以选择运用该框架实现一些简单的应用,或者可以深入研究其源码,提升自身的编程水平 在初识Bottle(一)中,我们了解了Bottle的基本用法在Bottle源码阅读(一)和Bottle源码阅读(二)可以查看个人对bottle源码的相关阅读笔记 下面继续阅读Bottle的官方文档https://bottlep...

    stormjun 评论0 收藏0
  • Bottle框架中的装饰器类和描述符应用

    摘要:最近在阅读微型框架的源码,发现了中有一个既是装饰器类又是描述符的有趣实现。所以第三版的代码可以这样写第三版的代码没有使用装饰器,而是使用了描述符这个技巧。更大的问题来自如何将描述符与装饰器结合起来,因为是一个类而不是方法。 最近在阅读Python微型Web框架Bottle的源码,发现了Bottle中有一个既是装饰器类又是描述符的有趣实现。刚好这两个点是Python比较的难理解,又混合在...

    Panda 评论0 收藏0
  • flask 源码解析:简介

    摘要:简介官网上对它的定位是一个微开发框架。另外一个必须理解的概念是,简单来说就是一套和框架应用之间的协议。功能比较丰富,支持解析自动防止攻击继承变量过滤器流程逻辑支持代码逻辑集成等等。那么,从下一篇文章,我们就正式开始源码之旅了 文章属于作者原创,原文发布在个人博客。 flask 简介 Flask 官网上对它的定位是一个微 python web 开发框架。 Flask is a micro...

    megatron 评论0 收藏0
  • 初识 Bottle

    摘要:安装是一个轻量型的不依赖于任何第三方库的框架,整个框架只有一个文件。向打声招呼吧新建一个文件在浏览器或者,,得到结果当使用装饰器绑定路由时,实际是使用了的默认应用,即是的一个实例。 1. 安装 bottle是一个轻量型的不依赖于任何第三方库的web框架,整个框架只有bottle.py一个文件。 wget http://bottlepy.org/bottle.py 2. 向bottl...

    mengbo 评论0 收藏0

发表评论

0条评论

whidy

|高级讲师

TA的文章

阅读更多
最新活动
阅读需要支付1元查看
<