Shared Memory Dictionary utilizing Posix IPC semaphores and shared memory segments and offering permanent disk storage of data if required.
Ви не можете вибрати більше 25 тем Теми мають розпочинатися з літери або цифри, можуть містити дефіси (-) і не повинні перевищувати 35 символів.

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307
  1. <p class="pln" id="t1"><span class="com">#!/usr/bin/env python3</span><span class="strut"> </span></p>
  2. <p class="pln" id="t2"><span class="com"># -*- coding: utf-8 -*-</span><span class="strut"> </span></p>
  3. <p class="pln" id="t3"><span class="strut"> </span></p>
  4. <p class="pln" id="t4"><span class="com"># Standard library imports</span><span class="strut"> </span></p>
  5. <p class="stm run hide_run" id="t5"><span class="key">import</span> <span class="nam">base64</span><span class="strut"> </span></p>
  6. <p class="pln" id="t6"><span class="strut"> </span></p>
  7. <p class="stm run hide_run" id="t7"><span class="key">try</span><span class="op">:</span><span class="strut"> </span></p>
  8. <p class="stm run hide_run" id="t8"> <span class="key">from</span> <span class="nam">collections</span><span class="op">.</span><span class="nam">abc</span> <span class="key">import</span> <span class="nam">MutableMapping</span><span class="strut"> </span></p>
  9. <p class="stm run hide_run" id="t9"><span class="key">except</span> <span class="nam">ImportError</span><span class="op">:</span><span class="strut"> </span></p>
  10. <p class="stm run hide_run" id="t10"> <span class="key">from</span> <span class="nam">collections</span> <span class="key">import</span> <span class="nam">MutableMapping</span><span class="strut"> </span></p>
  11. <p class="pln" id="t11"><span class="strut"> </span></p>
  12. <p class="stm run hide_run" id="t12"><span class="key">from</span> <span class="nam">contextlib</span> <span class="key">import</span> <span class="nam">contextmanager</span><span class="strut"> </span></p>
  13. <p class="stm run hide_run" id="t13"><span class="key">import</span> <span class="nam">hashlib</span><span class="strut"> </span></p>
  14. <p class="stm run hide_run" id="t14"><span class="key">import</span> <span class="nam">logging</span><span class="strut"> </span></p>
  15. <p class="stm run hide_run" id="t15"><span class="key">from</span> <span class="nam">math</span> <span class="key">import</span> <span class="nam">ceil</span><span class="strut"> </span></p>
  16. <p class="stm run hide_run" id="t16"><span class="key">import</span> <span class="nam">mmap</span><span class="strut"> </span></p>
  17. <p class="stm run hide_run" id="t17"><span class="key">import</span> <span class="nam">os</span><span class="strut"> </span></p>
  18. <p class="stm run hide_run" id="t18"><span class="key">import</span> <span class="nam">pickle</span> <span class="com"># nosec</span><span class="strut"> </span></p>
  19. <p class="stm run hide_run" id="t19"><span class="key">import</span> <span class="nam">sys</span><span class="strut"> </span></p>
  20. <p class="stm run hide_run" id="t20"><span class="key">import</span> <span class="nam">threading</span><span class="strut"> </span></p>
  21. <p class="pln" id="t21"><span class="strut"> </span></p>
  22. <p class="pln" id="t22"><span class="com"># Related third party imports (If you used pip/apt/yum to install)</span><span class="strut"> </span></p>
  23. <p class="stm run hide_run" id="t23"><span class="key">import</span> <span class="nam">posix_ipc</span><span class="strut"> </span></p>
  24. <p class="stm run hide_run" id="t24"><span class="key">import</span> <span class="nam">six</span><span class="strut"> </span></p>
  25. <p class="pln" id="t25"><span class="strut"> </span></p>
  26. <p class="pln" id="t26"><span class="com"># Local application/library specific imports (Look ma! I wrote it myself!)</span><span class="strut"> </span></p>
  27. <p class="stm run hide_run" id="t27"><span class="key">from</span> <span class="op">.</span><span class="nam">_version</span> <span class="key">import</span> <span class="nam">__version__</span><span class="strut"> </span></p>
  28. <p class="pln" id="t28"><span class="strut"> </span></p>
  29. <p class="stm run hide_run" id="t29"><span class="nam">__author__</span> <span class="op">=</span> <span class="str">"Nate Bohman"</span><span class="strut"> </span></p>
  30. <p class="stm run hide_run" id="t30"><span class="nam">__credits__</span> <span class="op">=</span> <span class="op">[</span><span class="str">"Nate Bohman"</span><span class="op">]</span><span class="strut"> </span></p>
  31. <p class="stm run hide_run" id="t31"><span class="nam">__license__</span> <span class="op">=</span> <span class="str">"LGPL-3"</span><span class="strut"> </span></p>
  32. <p class="stm run hide_run" id="t32"><span class="nam">__maintainer__</span> <span class="op">=</span> <span class="str">"Nate Bohman"</span><span class="strut"> </span></p>
  33. <p class="stm run hide_run" id="t33"><span class="nam">__email__</span> <span class="op">=</span> <span class="str">"natrinicle-shm_dict@natrinicle.com"</span><span class="strut"> </span></p>
  34. <p class="stm run hide_run" id="t34"><span class="nam">__status__</span> <span class="op">=</span> <span class="str">"Production"</span><span class="strut"> </span></p>
  35. <p class="pln" id="t35"><span class="strut"> </span></p>
  36. <p class="stm run hide_run" id="t36"><span class="nam">logger</span> <span class="op">=</span> <span class="nam">logging</span><span class="op">.</span><span class="nam">getLogger</span><span class="op">(</span><span class="nam">__name__</span><span class="op">)</span> <span class="com"># pylint: disable=invalid-name</span><span class="strut"> </span></p>
  37. <p class="pln" id="t37"><span class="strut"> </span></p>
  38. <p class="pln" id="t38"><span class="strut"> </span></p>
  39. <p class="stm run hide_run" id="t39"><span class="key">class</span> <span class="nam">SHMDict</span><span class="op">(</span><span class="nam">MutableMapping</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  40. <p class="pln" id="t40"> <span class="str">"""Python shared memory dictionary."""</span><span class="strut"> </span></p>
  41. <p class="pln" id="t41"><span class="strut"> </span></p>
  42. <p class="stm run hide_run" id="t42"> <span class="key">def</span> <span class="nam">__init__</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">name</span><span class="op">,</span> <span class="nam">persist</span><span class="op">=</span><span class="key">False</span><span class="op">,</span> <span class="nam">lock_timeout</span><span class="op">=</span><span class="num">30</span><span class="op">,</span> <span class="nam">auto_unlock</span><span class="op">=</span><span class="key">False</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  43. <p class="pln" id="t43"> <span class="str">"""Standard init method.</span><span class="strut"> </span></p>
  44. <p class="pln" id="t44"><span class="strut"> </span></p>
  45. <p class="pln" id="t45"><span class="str"> :param name: Name for shared memory and semaphore if volatile</span><span class="strut"> </span></p>
  46. <p class="pln" id="t46"><span class="str"> or path to file if persistent.</span><span class="strut"> </span></p>
  47. <p class="pln" id="t47"><span class="str"> :param persist: True if name is the path to a file and this</span><span class="strut"> </span></p>
  48. <p class="pln" id="t48"><span class="str"> shared memory dictionary should be written</span><span class="strut"> </span></p>
  49. <p class="pln" id="t49"><span class="str"> out to the file for persistence between runs</span><span class="strut"> </span></p>
  50. <p class="pln" id="t50"><span class="str"> and/or processes.</span><span class="strut"> </span></p>
  51. <p class="pln" id="t51"><span class="str"> :param lock_timeout: Time in seconds before giving up on</span><span class="strut"> </span></p>
  52. <p class="pln" id="t52"><span class="str"> acquiring an exclusive lock to the</span><span class="strut"> </span></p>
  53. <p class="pln" id="t53"><span class="str"> dictionary.</span><span class="strut"> </span></p>
  54. <p class="pln" id="t54"><span class="str"> :param auto_unlock: If the lock_timeout is hit, and this</span><span class="strut"> </span></p>
  55. <p class="pln" id="t55"><span class="str"> is True, automatically bypass the</span><span class="strut"> </span></p>
  56. <p class="pln" id="t56"><span class="str"> lock and use the dictionary anyway.</span><span class="strut"> </span></p>
  57. <p class="pln" id="t57"><span class="str"> :type name: :class:`str`</span><span class="strut"> </span></p>
  58. <p class="pln" id="t58"><span class="str"> :type persist: :class:`bool`</span><span class="strut"> </span></p>
  59. <p class="pln" id="t59"><span class="str"> :type lock_timeout: :class:`int` or :class:`float`</span><span class="strut"> </span></p>
  60. <p class="pln" id="t60"><span class="str"> :type auto_unlock: :class:`bool`</span><span class="strut"> </span></p>
  61. <p class="pln" id="t61"><span class="str"> """</span><span class="strut"> </span></p>
  62. <p class="stm run hide_run" id="t62"> <span class="nam">self</span><span class="op">.</span><span class="nam">name</span> <span class="op">=</span> <span class="nam">name</span><span class="strut"> </span></p>
  63. <p class="stm run hide_run" id="t63"> <span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span> <span class="op">=</span> <span class="key">None</span><span class="strut"> </span></p>
  64. <p class="stm run hide_run" id="t64"> <span class="nam">self</span><span class="op">.</span><span class="nam">lock_timeout</span> <span class="op">=</span> <span class="nam">lock_timeout</span><span class="strut"> </span></p>
  65. <p class="stm run hide_run" id="t65"> <span class="nam">self</span><span class="op">.</span><span class="nam">auto_unlock</span> <span class="op">=</span> <span class="nam">auto_unlock</span><span class="strut"> </span></p>
  66. <p class="stm run hide_run" id="t66"> <span class="nam">self</span><span class="op">.</span><span class="nam">_semaphore</span> <span class="op">=</span> <span class="key">None</span><span class="strut"> </span></p>
  67. <p class="stm run hide_run" id="t67"> <span class="nam">self</span><span class="op">.</span><span class="nam">_map_file</span> <span class="op">=</span> <span class="key">None</span><span class="strut"> </span></p>
  68. <p class="stm run hide_run" id="t68"> <span class="nam">self</span><span class="op">.</span><span class="nam">__thread_local</span> <span class="op">=</span> <span class="nam">threading</span><span class="op">.</span><span class="nam">local</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  69. <p class="stm run hide_run" id="t69"> <span class="nam">self</span><span class="op">.</span><span class="nam">__thread_local</span><span class="op">.</span><span class="nam">semaphore</span> <span class="op">=</span> <span class="key">False</span><span class="strut"> </span></p>
  70. <p class="stm run hide_run" id="t70"> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span> <span class="op">=</span> <span class="key">None</span><span class="strut"> </span></p>
  71. <p class="stm run hide_run" id="t71"> <span class="nam">self</span><span class="op">.</span><span class="nam">__dirty</span> <span class="op">=</span> <span class="key">False</span><span class="strut"> </span></p>
  72. <p class="pln" id="t72"><span class="strut"> </span></p>
  73. <p class="stm run hide_run" id="t73"> <span class="key">if</span> <span class="nam">persist</span> <span class="key">is</span> <span class="key">True</span><span class="op">:</span><span class="strut"> </span></p>
  74. <p class="stm run hide_run" id="t74"> <span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span> <span class="op">=</span> <span class="nam">self</span><span class="op">.</span><span class="nam">name</span><span class="strut"> </span></p>
  75. <p class="stm run hide_run" id="t75"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span><span class="op">.</span><span class="nam">startswith</span><span class="op">(</span><span class="str">"~"</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  76. <p class="stm run hide_run" id="t76"> <span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span> <span class="op">=</span> <span class="nam">os</span><span class="op">.</span><span class="nam">path</span><span class="op">.</span><span class="nam">expanduser</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span><span class="op">)</span><span class="strut"> </span></p>
  77. <p class="stm run hide_run" id="t77"> <span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span> <span class="op">=</span> <span class="nam">os</span><span class="op">.</span><span class="nam">path</span><span class="op">.</span><span class="nam">abspath</span><span class="op">(</span><span class="nam">os</span><span class="op">.</span><span class="nam">path</span><span class="op">.</span><span class="nam">realpath</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span><span class="op">)</span><span class="op">)</span><span class="strut"> </span></p>
  78. <p class="pln" id="t78"><span class="strut"> </span></p>
  79. <p class="stm run hide_run" id="t79"> <span class="nam">super</span><span class="op">(</span><span class="nam">SHMDict</span><span class="op">,</span> <span class="nam">self</span><span class="op">)</span><span class="op">.</span><span class="nam">__init__</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  80. <p class="pln" id="t80"><span class="strut"> </span></p>
  81. <p class="stm run hide_run" id="t81"> <span class="key">def</span> <span class="nam">_safe_name</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">prefix</span><span class="op">=</span><span class="str">""</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  82. <p class="pln" id="t82"> <span class="str">"""IPC object safe name creator.</span><span class="strut"> </span></p>
  83. <p class="pln" id="t83"><span class="strut"> </span></p>
  84. <p class="pln" id="t84"><span class="str"> Semaphores and Shared Mmeory names allow up to 256 characters (dependong on OS) and must</span><span class="strut"> </span></p>
  85. <p class="pln" id="t85"><span class="str"> begin with a /.</span><span class="strut"> </span></p>
  86. <p class="pln" id="t86"><span class="strut"> </span></p>
  87. <p class="pln" id="t87"><span class="str"> :param prefix: A string to prepend followed by _ and</span><span class="strut"> </span></p>
  88. <p class="pln" id="t88"><span class="str"> then the dictionary's name.</span><span class="strut"> </span></p>
  89. <p class="pln" id="t89"><span class="str"> :type prefix: :class:`str`</span><span class="strut"> </span></p>
  90. <p class="pln" id="t90"><span class="str"> """</span><span class="strut"> </span></p>
  91. <p class="pln" id="t91"> <span class="com"># Hash lengths</span><span class="strut"> </span></p>
  92. <p class="pln" id="t92"> <span class="com"># SHA1: 28</span><span class="strut"> </span></p>
  93. <p class="pln" id="t93"> <span class="com"># SHA256: 44</span><span class="strut"> </span></p>
  94. <p class="pln" id="t94"> <span class="com"># SHA512: 88</span><span class="strut"> </span></p>
  95. <p class="stm run hide_run" id="t95"> <span class="nam">sha_hash</span> <span class="op">=</span> <span class="nam">hashlib</span><span class="op">.</span><span class="nam">sha512</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  96. <p class="stm run hide_run" id="t96"> <span class="nam">sha_hash</span><span class="op">.</span><span class="nam">update</span><span class="op">(</span><span class="str">"_"</span><span class="op">.</span><span class="nam">join</span><span class="op">(</span><span class="op">[</span><span class="nam">prefix</span><span class="op">,</span> <span class="nam">str</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">name</span><span class="op">)</span><span class="op">]</span><span class="op">)</span><span class="op">.</span><span class="nam">encode</span><span class="op">(</span><span class="str">"utf-8"</span><span class="op">)</span><span class="op">)</span><span class="strut"> </span></p>
  97. <p class="stm run hide_run" id="t97"> <span class="nam">b64_encode</span> <span class="op">=</span> <span class="nam">base64</span><span class="op">.</span><span class="nam">urlsafe_b64encode</span><span class="op">(</span><span class="nam">sha_hash</span><span class="op">.</span><span class="nam">digest</span><span class="op">(</span><span class="op">)</span><span class="op">)</span><span class="strut"> </span></p>
  98. <p class="stm run hide_run" id="t98"> <span class="key">return</span> <span class="str">"/{}"</span><span class="op">.</span><span class="nam">format</span><span class="op">(</span><span class="nam">b64_encode</span><span class="op">)</span><span class="strut"> </span></p>
  99. <p class="pln" id="t99"><span class="strut"> </span></p>
  100. <p class="stm run hide_run" id="t100"> <span class="op">@</span><span class="nam">property</span><span class="strut"> </span></p>
  101. <p class="pln" id="t101"> <span class="key">def</span> <span class="nam">safe_sem_name</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  102. <p class="pln" id="t102"> <span class="str">"""Unique semaphore name based on the dictionary name."""</span><span class="strut"> </span></p>
  103. <p class="stm run hide_run" id="t103"> <span class="key">return</span> <span class="nam">self</span><span class="op">.</span><span class="nam">_safe_name</span><span class="op">(</span><span class="str">"sem"</span><span class="op">)</span><span class="strut"> </span></p>
  104. <p class="pln" id="t104"><span class="strut"> </span></p>
  105. <p class="stm run hide_run" id="t105"> <span class="op">@</span><span class="nam">property</span><span class="strut"> </span></p>
  106. <p class="pln" id="t106"> <span class="key">def</span> <span class="nam">safe_shm_name</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  107. <p class="pln" id="t107"> <span class="str">"""Unique shared memory segment name based on the dictionary name."""</span><span class="strut"> </span></p>
  108. <p class="stm run hide_run" id="t108"> <span class="key">return</span> <span class="nam">self</span><span class="op">.</span><span class="nam">_safe_name</span><span class="op">(</span><span class="str">"shm"</span><span class="op">)</span><span class="strut"> </span></p>
  109. <p class="pln" id="t109"><span class="strut"> </span></p>
  110. <p class="stm run hide_run" id="t110"> <span class="op">@</span><span class="nam">property</span><span class="strut"> </span></p>
  111. <p class="pln" id="t111"> <span class="key">def</span> <span class="nam">semaphore</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  112. <p class="pln" id="t112"> <span class="str">"""Create or return already existing semaphore."""</span><span class="strut"> </span></p>
  113. <p class="stm run hide_run" id="t113"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">_semaphore</span> <span class="key">is</span> <span class="key">not</span> <span class="key">None</span><span class="op">:</span><span class="strut"> </span></p>
  114. <p class="stm run hide_run" id="t114"> <span class="key">return</span> <span class="nam">self</span><span class="op">.</span><span class="nam">_semaphore</span><span class="strut"> </span></p>
  115. <p class="pln" id="t115"><span class="strut"> </span></p>
  116. <p class="stm run hide_run" id="t116"> <span class="key">try</span><span class="op">:</span><span class="strut"> </span></p>
  117. <p class="stm run hide_run" id="t117"> <span class="nam">self</span><span class="op">.</span><span class="nam">_semaphore</span> <span class="op">=</span> <span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">Semaphore</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">safe_sem_name</span><span class="op">)</span><span class="strut"> </span></p>
  118. <p class="stm run hide_run" id="t118"> <span class="key">except</span> <span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">ExistentialError</span><span class="op">:</span><span class="strut"> </span></p>
  119. <p class="stm run hide_run" id="t119"> <span class="nam">self</span><span class="op">.</span><span class="nam">_semaphore</span> <span class="op">=</span> <span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">Semaphore</span><span class="op">(</span><span class="strut"> </span></p>
  120. <p class="pln" id="t120"> <span class="nam">self</span><span class="op">.</span><span class="nam">safe_sem_name</span><span class="op">,</span> <span class="nam">flags</span><span class="op">=</span><span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">O_CREAT</span><span class="op">,</span> <span class="nam">initial_value</span><span class="op">=</span><span class="num">1</span><span class="strut"> </span></p>
  121. <p class="pln" id="t121"> <span class="op">)</span><span class="strut"> </span></p>
  122. <p class="stm run hide_run" id="t122"> <span class="key">return</span> <span class="nam">self</span><span class="op">.</span><span class="nam">_semaphore</span><span class="strut"> </span></p>
  123. <p class="pln" id="t123"><span class="strut"> </span></p>
  124. <p class="stm run hide_run" id="t124"> <span class="op">@</span><span class="nam">property</span><span class="strut"> </span></p>
  125. <p class="pln" id="t125"> <span class="key">def</span> <span class="nam">shared_mem</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  126. <p class="pln" id="t126"> <span class="str">"""Create or return already existing shared memory object."""</span><span class="strut"> </span></p>
  127. <p class="stm run hide_run" id="t127"> <span class="key">try</span><span class="op">:</span><span class="strut"> </span></p>
  128. <p class="stm run hide_run" id="t128"> <span class="key">return</span> <span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">SharedMemory</span><span class="op">(</span><span class="strut"> </span></p>
  129. <p class="pln" id="t129"> <span class="nam">self</span><span class="op">.</span><span class="nam">safe_shm_name</span><span class="op">,</span> <span class="nam">size</span><span class="op">=</span><span class="nam">len</span><span class="op">(</span><span class="nam">pickle</span><span class="op">.</span><span class="nam">dumps</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">)</span><span class="op">)</span><span class="strut"> </span></p>
  130. <p class="pln" id="t130"> <span class="op">)</span><span class="strut"> </span></p>
  131. <p class="stm run hide_run" id="t131"> <span class="key">except</span> <span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">ExistentialError</span><span class="op">:</span><span class="strut"> </span></p>
  132. <p class="stm run hide_run" id="t132"> <span class="key">return</span> <span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">SharedMemory</span><span class="op">(</span><span class="strut"> </span></p>
  133. <p class="pln" id="t133"> <span class="nam">self</span><span class="op">.</span><span class="nam">safe_shm_name</span><span class="op">,</span> <span class="nam">flags</span><span class="op">=</span><span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">O_CREX</span><span class="op">,</span> <span class="nam">size</span><span class="op">=</span><span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">PAGE_SIZE</span><span class="strut"> </span></p>
  134. <p class="pln" id="t134"> <span class="op">)</span><span class="strut"> </span></p>
  135. <p class="pln" id="t135"><span class="strut"> </span></p>
  136. <p class="stm run hide_run" id="t136"> <span class="op">@</span><span class="nam">property</span><span class="strut"> </span></p>
  137. <p class="pln" id="t137"> <span class="key">def</span> <span class="nam">map_file</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  138. <p class="pln" id="t138"> <span class="str">"""Create or return mmap file resizing if necessary."""</span><span class="strut"> </span></p>
  139. <p class="stm run hide_run" id="t139"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">_map_file</span> <span class="key">is</span> <span class="key">None</span><span class="op">:</span><span class="strut"> </span></p>
  140. <p class="stm run hide_run" id="t140"> <span class="nam">self</span><span class="op">.</span><span class="nam">_map_file</span> <span class="op">=</span> <span class="nam">mmap</span><span class="op">.</span><span class="nam">mmap</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">shared_mem</span><span class="op">.</span><span class="nam">fd</span><span class="op">,</span> <span class="nam">self</span><span class="op">.</span><span class="nam">shared_mem</span><span class="op">.</span><span class="nam">size</span><span class="op">)</span><span class="strut"> </span></p>
  141. <p class="stm run hide_run" id="t141"> <span class="nam">self</span><span class="op">.</span><span class="nam">shared_mem</span><span class="op">.</span><span class="nam">close_fd</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  142. <p class="pln" id="t142"><span class="strut"> </span></p>
  143. <p class="stm run hide_run" id="t143"> <span class="nam">self</span><span class="op">.</span><span class="nam">_map_file</span><span class="op">.</span><span class="nam">resize</span><span class="op">(</span><span class="strut"> </span></p>
  144. <p class="pln" id="t144"> <span class="nam">int</span><span class="op">(</span><span class="strut"> </span></p>
  145. <p class="pln" id="t145"> <span class="nam">ceil</span><span class="op">(</span><span class="nam">float</span><span class="op">(</span><span class="nam">len</span><span class="op">(</span><span class="nam">pickle</span><span class="op">.</span><span class="nam">dumps</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">,</span> <span class="num">2</span><span class="op">)</span><span class="op">)</span><span class="op">)</span> <span class="op">/</span> <span class="nam">mmap</span><span class="op">.</span><span class="nam">PAGESIZE</span><span class="op">)</span><span class="strut"> </span></p>
  146. <p class="pln" id="t146"> <span class="op">*</span> <span class="nam">mmap</span><span class="op">.</span><span class="nam">PAGESIZE</span><span class="strut"> </span></p>
  147. <p class="pln" id="t147"> <span class="op">)</span><span class="strut"> </span></p>
  148. <p class="pln" id="t148"> <span class="op">)</span><span class="strut"> </span></p>
  149. <p class="stm run hide_run" id="t149"> <span class="key">return</span> <span class="nam">self</span><span class="op">.</span><span class="nam">_map_file</span><span class="strut"> </span></p>
  150. <p class="pln" id="t150"><span class="strut"> </span></p>
  151. <p class="stm run hide_run" id="t151"> <span class="key">def</span> <span class="nam">__load_dict</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  152. <p class="pln" id="t152"> <span class="str">"""Load dictionary from shared memory or file if persistent and memory empty."""</span><span class="strut"> </span></p>
  153. <p class="pln" id="t153"> <span class="com"># Read in internal data from map_file</span><span class="strut"> </span></p>
  154. <p class="stm run hide_run" id="t154"> <span class="nam">self</span><span class="op">.</span><span class="nam">map_file</span><span class="op">.</span><span class="nam">seek</span><span class="op">(</span><span class="num">0</span><span class="op">)</span><span class="strut"> </span></p>
  155. <p class="stm run hide_run" id="t155"> <span class="key">try</span><span class="op">:</span><span class="strut"> </span></p>
  156. <p class="stm run hide_run" id="t156"> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span> <span class="op">=</span> <span class="nam">pickle</span><span class="op">.</span><span class="nam">load</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">map_file</span><span class="op">)</span> <span class="com"># nosec</span><span class="strut"> </span></p>
  157. <p class="stm run hide_run" id="t157"> <span class="key">except</span> <span class="op">(</span><span class="nam">KeyError</span><span class="op">,</span> <span class="nam">pickle</span><span class="op">.</span><span class="nam">UnpicklingError</span><span class="op">,</span> <span class="nam">EOFError</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  158. <p class="pln" id="t158"> <span class="com"># Curtis Pullen found that Python 3.4 throws EOFError</span><span class="strut"> </span></p>
  159. <p class="pln" id="t159"> <span class="com"># instead of UnpicklingError that Python 3.6 throws</span><span class="strut"> </span></p>
  160. <p class="pln" id="t160"> <span class="com"># when attempting to unpickle an empty file.</span><span class="strut"> </span></p>
  161. <p class="stm run hide_run" id="t161"> <span class="key">pass</span><span class="strut"> </span></p>
  162. <p class="pln" id="t162"><span class="strut"> </span></p>
  163. <p class="pln" id="t163"> <span class="com"># If map_file is empty and persist_file is true, treat</span><span class="strut"> </span></p>
  164. <p class="pln" id="t164"> <span class="com"># self.name as filename and attempt to load from disk.</span><span class="strut"> </span></p>
  165. <p class="stm run hide_run" id="t165"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span> <span class="key">is</span> <span class="key">None</span> <span class="key">and</span> <span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span> <span class="key">is</span> <span class="key">not</span> <span class="key">None</span><span class="op">:</span><span class="strut"> </span></p>
  166. <p class="stm run hide_run" id="t166"> <span class="key">try</span><span class="op">:</span><span class="strut"> </span></p>
  167. <p class="stm run hide_run" id="t167"> <span class="key">with</span> <span class="nam">open</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span><span class="op">,</span> <span class="str">"rb"</span><span class="op">)</span> <span class="key">as</span> <span class="nam">pfile</span><span class="op">:</span><span class="strut"> </span></p>
  168. <p class="stm run hide_run" id="t168"> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span> <span class="op">=</span> <span class="nam">pickle</span><span class="op">.</span><span class="nam">load</span><span class="op">(</span><span class="nam">pfile</span><span class="op">)</span> <span class="com"># nosec</span><span class="strut"> </span></p>
  169. <p class="stm run hide_run" id="t169"> <span class="key">except</span> <span class="nam">IOError</span><span class="op">:</span><span class="strut"> </span></p>
  170. <p class="stm run hide_run" id="t170"> <span class="key">pass</span><span class="strut"> </span></p>
  171. <p class="pln" id="t171"><span class="strut"> </span></p>
  172. <p class="pln" id="t172"> <span class="com"># If map_file is empty, persist_file is False or</span><span class="strut"> </span></p>
  173. <p class="pln" id="t173"> <span class="com"># self.name is empty create a new empty dictionary.</span><span class="strut"> </span></p>
  174. <p class="stm run hide_run" id="t174"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span> <span class="key">is</span> <span class="key">None</span><span class="op">:</span><span class="strut"> </span></p>
  175. <p class="stm run hide_run" id="t175"> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span> <span class="op">=</span> <span class="op">{</span><span class="op">}</span><span class="strut"> </span></p>
  176. <p class="pln" id="t176"><span class="strut"> </span></p>
  177. <p class="stm run hide_run" id="t177"> <span class="key">def</span> <span class="nam">__save_dict</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  178. <p class="pln" id="t178"> <span class="str">"""Save dictionary into shared memory and file if persistent."""</span><span class="strut"> </span></p>
  179. <p class="pln" id="t179"> <span class="com"># Write out internal dict to map_file</span><span class="strut"> </span></p>
  180. <p class="stm run hide_run" id="t180"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__dirty</span> <span class="key">is</span> <span class="key">True</span><span class="op">:</span><span class="strut"> </span></p>
  181. <p class="stm run hide_run" id="t181"> <span class="nam">self</span><span class="op">.</span><span class="nam">map_file</span><span class="op">.</span><span class="nam">seek</span><span class="op">(</span><span class="num">0</span><span class="op">)</span><span class="strut"> </span></p>
  182. <p class="stm run hide_run" id="t182"> <span class="nam">pickle</span><span class="op">.</span><span class="nam">dump</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">,</span> <span class="nam">self</span><span class="op">.</span><span class="nam">map_file</span><span class="op">,</span> <span class="num">2</span><span class="op">)</span><span class="strut"> </span></p>
  183. <p class="pln" id="t183"><span class="strut"> </span></p>
  184. <p class="stm run hide_run" id="t184"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span> <span class="key">is</span> <span class="key">not</span> <span class="key">None</span><span class="op">:</span><span class="strut"> </span></p>
  185. <p class="stm run hide_run" id="t185"> <span class="key">with</span> <span class="nam">open</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">persist_file</span><span class="op">,</span> <span class="str">"wb"</span><span class="op">)</span> <span class="key">as</span> <span class="nam">pfile</span><span class="op">:</span><span class="strut"> </span></p>
  186. <p class="stm run hide_run" id="t186"> <span class="nam">pickle</span><span class="op">.</span><span class="nam">dump</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">,</span> <span class="nam">pfile</span><span class="op">,</span> <span class="num">2</span><span class="op">)</span><span class="strut"> </span></p>
  187. <p class="pln" id="t187"><span class="strut"> </span></p>
  188. <p class="stm run hide_run" id="t188"> <span class="nam">self</span><span class="op">.</span><span class="nam">__dirty</span> <span class="op">=</span> <span class="key">False</span><span class="strut"> </span></p>
  189. <p class="pln" id="t189"><span class="strut"> </span></p>
  190. <p class="stm run hide_run" id="t190"> <span class="key">def</span> <span class="nam">_acquire_lock</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  191. <p class="pln" id="t191"> <span class="str">"""Acquire an exclusive dict lock.</span><span class="strut"> </span></p>
  192. <p class="pln" id="t192"><span class="strut"> </span></p>
  193. <p class="pln" id="t193"><span class="str"> Loads dictionary data from memory or disk (if persistent) to</span><span class="strut"> </span></p>
  194. <p class="pln" id="t194"><span class="str"> ensure data is up to date when lock is requested.</span><span class="strut"> </span></p>
  195. <p class="pln" id="t195"><span class="strut"> </span></p>
  196. <p class="pln" id="t196"><span class="str"> .. warnings also::</span><span class="strut"> </span></p>
  197. <p class="pln" id="t197"><span class="str"> MacOS has a number of shortcomings with regards to</span><span class="strut"> </span></p>
  198. <p class="pln" id="t198"><span class="str"> semaphores and shared memory segments, this is one</span><span class="strut"> </span></p>
  199. <p class="pln" id="t199"><span class="str"> method contains one of them.</span><span class="strut"> </span></p>
  200. <p class="pln" id="t200"><span class="strut"> </span></p>
  201. <p class="pln" id="t201"><span class="str"> When the timeout is &gt; 0, the call will wait no longer than</span><span class="strut"> </span></p>
  202. <p class="pln" id="t202"><span class="str"> timeout seconds before either returning (having acquired</span><span class="strut"> </span></p>
  203. <p class="pln" id="t203"><span class="str"> the semaphore) or raising a BusyError.</span><span class="strut"> </span></p>
  204. <p class="pln" id="t204"><span class="str"> On platforms that don't support the sem_timedwait() API,</span><span class="strut"> </span></p>
  205. <p class="pln" id="t205"><span class="str"> a timeout &gt; 0 is treated as infinite. The call will not</span><span class="strut"> </span></p>
  206. <p class="pln" id="t206"><span class="str"> return until its wait condition is satisfied.</span><span class="strut"> </span></p>
  207. <p class="pln" id="t207"><span class="str"> Most platforms provide sem_timedwait(). macOS is a notable</span><span class="strut"> </span></p>
  208. <p class="pln" id="t208"><span class="str"> exception. The module's Boolean constant</span><span class="strut"> </span></p>
  209. <p class="pln" id="t209"><span class="str"> SEMAPHORE_TIMEOUT_SUPPORTED is True on platforms that</span><span class="strut"> </span></p>
  210. <p class="pln" id="t210"><span class="str"> support sem_timedwait().</span><span class="strut"> </span></p>
  211. <p class="pln" id="t211"><span class="strut"> </span></p>
  212. <p class="pln" id="t212"><span class="str"> -- http://semanchuk.com/philip/posix_ipc/</span><span class="strut"> </span></p>
  213. <p class="pln" id="t213"><span class="str"> """</span><span class="strut"> </span></p>
  214. <p class="stm run hide_run" id="t214"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__thread_local</span><span class="op">.</span><span class="nam">semaphore</span> <span class="key">is</span> <span class="key">False</span><span class="op">:</span><span class="strut"> </span></p>
  215. <p class="stm run hide_run" id="t215"> <span class="key">try</span><span class="op">:</span><span class="strut"> </span></p>
  216. <p class="stm run hide_run" id="t216"> <span class="nam">self</span><span class="op">.</span><span class="nam">semaphore</span><span class="op">.</span><span class="nam">acquire</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">lock_timeout</span><span class="op">)</span><span class="strut"> </span></p>
  217. <p class="stm run hide_run" id="t217"> <span class="nam">self</span><span class="op">.</span><span class="nam">__thread_local</span><span class="op">.</span><span class="nam">semaphore</span> <span class="op">=</span> <span class="key">True</span><span class="strut"> </span></p>
  218. <p class="stm run hide_run" id="t218"> <span class="key">except</span> <span class="nam">posix_ipc</span><span class="op">.</span><span class="nam">BusyError</span><span class="op">:</span><span class="strut"> </span></p>
  219. <p class="stm run hide_run" id="t219"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">auto_unlock</span> <span class="key">is</span> <span class="key">True</span><span class="op">:</span><span class="strut"> </span></p>
  220. <p class="stm run hide_run" id="t220"> <span class="nam">self</span><span class="op">.</span><span class="nam">__thread_local</span><span class="op">.</span><span class="nam">semaphore</span> <span class="op">=</span> <span class="key">True</span><span class="strut"> </span></p>
  221. <p class="pln" id="t221"> <span class="key">else</span><span class="op">:</span><span class="strut"> </span></p>
  222. <p class="stm run hide_run" id="t222"> <span class="nam">six</span><span class="op">.</span><span class="nam">reraise</span><span class="op">(</span><span class="op">*</span><span class="nam">sys</span><span class="op">.</span><span class="nam">exc_info</span><span class="op">(</span><span class="op">)</span><span class="op">)</span><span class="strut"> </span></p>
  223. <p class="pln" id="t223"><span class="strut"> </span></p>
  224. <p class="stm run hide_run" id="t224"> <span class="nam">self</span><span class="op">.</span><span class="nam">__load_dict</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  225. <p class="pln" id="t225"><span class="strut"> </span></p>
  226. <p class="stm run hide_run" id="t226"> <span class="key">def</span> <span class="nam">_release_lock</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  227. <p class="pln" id="t227"> <span class="str">"""Release the exclusive semaphore lock."""</span><span class="strut"> </span></p>
  228. <p class="stm run hide_run" id="t228"> <span class="key">if</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__thread_local</span><span class="op">.</span><span class="nam">semaphore</span> <span class="key">is</span> <span class="key">True</span><span class="op">:</span><span class="strut"> </span></p>
  229. <p class="stm run hide_run" id="t229"> <span class="nam">self</span><span class="op">.</span><span class="nam">__save_dict</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  230. <p class="stm run hide_run" id="t230"> <span class="nam">self</span><span class="op">.</span><span class="nam">semaphore</span><span class="op">.</span><span class="nam">release</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  231. <p class="stm run hide_run" id="t231"> <span class="nam">self</span><span class="op">.</span><span class="nam">__thread_local</span><span class="op">.</span><span class="nam">semaphore</span> <span class="op">=</span> <span class="key">False</span><span class="strut"> </span></p>
  232. <p class="pln" id="t232"><span class="strut"> </span></p>
  233. <p class="stm run hide_run" id="t233"> <span class="op">@</span><span class="nam">contextmanager</span><span class="strut"> </span></p>
  234. <p class="pln" id="t234"> <span class="key">def</span> <span class="nam">exclusive_lock</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  235. <p class="pln" id="t235"> <span class="str">"""A context manager for the lock to allow with statements for exclusive access."""</span><span class="strut"> </span></p>
  236. <p class="stm run hide_run" id="t236"> <span class="nam">self</span><span class="op">.</span><span class="nam">_acquire_lock</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  237. <p class="stm run hide_run" id="t237"> <span class="key">yield</span><span class="strut"> </span></p>
  238. <p class="stm run hide_run" id="t238"> <span class="nam">self</span><span class="op">.</span><span class="nam">_release_lock</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  239. <p class="pln" id="t239"><span class="strut"> </span></p>
  240. <p class="stm run hide_run" id="t240"> <span class="key">def</span> <span class="nam">__del__</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  241. <p class="pln" id="t241"> <span class="str">"""Destroy the object nicely."""</span><span class="strut"> </span></p>
  242. <p class="stm run hide_run" id="t242"> <span class="nam">self</span><span class="op">.</span><span class="nam">map_file</span><span class="op">.</span><span class="nam">close</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  243. <p class="stm run hide_run" id="t243"> <span class="nam">self</span><span class="op">.</span><span class="nam">shared_mem</span><span class="op">.</span><span class="nam">unlink</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  244. <p class="stm run hide_run" id="t244"> <span class="nam">self</span><span class="op">.</span><span class="nam">semaphore</span><span class="op">.</span><span class="nam">unlink</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  245. <p class="pln" id="t245"><span class="strut"> </span></p>
  246. <p class="stm run hide_run" id="t246"> <span class="key">def</span> <span class="nam">__setitem__</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">key</span><span class="op">,</span> <span class="nam">value</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  247. <p class="pln" id="t247"> <span class="str">"""Set a key in the dictionary to a value."""</span><span class="strut"> </span></p>
  248. <p class="stm run hide_run" id="t248"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  249. <p class="stm run hide_run" id="t249"> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">[</span><span class="nam">key</span><span class="op">]</span> <span class="op">=</span> <span class="nam">value</span><span class="strut"> </span></p>
  250. <p class="stm run hide_run" id="t250"> <span class="nam">self</span><span class="op">.</span><span class="nam">__dirty</span> <span class="op">=</span> <span class="key">True</span><span class="strut"> </span></p>
  251. <p class="pln" id="t251"><span class="strut"> </span></p>
  252. <p class="stm run hide_run" id="t252"> <span class="key">def</span> <span class="nam">__getitem__</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">key</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  253. <p class="pln" id="t253"> <span class="str">"""Get the value of a key from the dictionary."""</span><span class="strut"> </span></p>
  254. <p class="stm run hide_run" id="t254"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  255. <p class="stm run hide_run" id="t255"> <span class="key">return</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">[</span><span class="nam">key</span><span class="op">]</span><span class="strut"> </span></p>
  256. <p class="pln" id="t256"><span class="strut"> </span></p>
  257. <p class="stm run hide_run" id="t257"> <span class="key">def</span> <span class="nam">__repr__</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  258. <p class="pln" id="t258"> <span class="str">"""Represent the dictionary in a human readable format."""</span><span class="strut"> </span></p>
  259. <p class="stm run hide_run" id="t259"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  260. <p class="stm run hide_run" id="t260"> <span class="key">return</span> <span class="nam">repr</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">)</span><span class="strut"> </span></p>
  261. <p class="pln" id="t261"><span class="strut"> </span></p>
  262. <p class="stm run hide_run" id="t262"> <span class="key">def</span> <span class="nam">__len__</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  263. <p class="pln" id="t263"> <span class="str">"""Return the length of the dictionary."""</span><span class="strut"> </span></p>
  264. <p class="stm run hide_run" id="t264"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  265. <p class="stm run hide_run" id="t265"> <span class="key">return</span> <span class="nam">len</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">)</span><span class="strut"> </span></p>
  266. <p class="pln" id="t266"><span class="strut"> </span></p>
  267. <p class="stm run hide_run" id="t267"> <span class="key">def</span> <span class="nam">__delitem__</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">key</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  268. <p class="pln" id="t268"> <span class="str">"""Remove an item from the dictionary."""</span><span class="strut"> </span></p>
  269. <p class="stm run hide_run" id="t269"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  270. <p class="stm run hide_run" id="t270"> <span class="key">del</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">[</span><span class="nam">key</span><span class="op">]</span><span class="strut"> </span></p>
  271. <p class="stm run hide_run" id="t271"> <span class="nam">self</span><span class="op">.</span><span class="nam">__dirty</span> <span class="op">=</span> <span class="key">True</span><span class="strut"> </span></p>
  272. <p class="pln" id="t272"><span class="strut"> </span></p>
  273. <p class="stm run hide_run" id="t273"> <span class="key">def</span> <span class="nam">clear</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  274. <p class="pln" id="t274"> <span class="str">"""Completely clear the dictionary."""</span><span class="strut"> </span></p>
  275. <p class="stm run hide_run" id="t275"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  276. <p class="stm run hide_run" id="t276"> <span class="nam">self</span><span class="op">.</span><span class="nam">__dirty</span> <span class="op">=</span> <span class="key">True</span><span class="strut"> </span></p>
  277. <p class="stm run hide_run" id="t277"> <span class="key">return</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">.</span><span class="nam">clear</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  278. <p class="pln" id="t278"><span class="strut"> </span></p>
  279. <p class="stm run hide_run" id="t279"> <span class="key">def</span> <span class="nam">copy</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  280. <p class="pln" id="t280"> <span class="str">"""Create and return a copy of the internal dictionary."""</span><span class="strut"> </span></p>
  281. <p class="stm run hide_run" id="t281"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  282. <p class="stm run hide_run" id="t282"> <span class="key">return</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">.</span><span class="nam">copy</span><span class="op">(</span><span class="op">)</span><span class="strut"> </span></p>
  283. <p class="pln" id="t283"><span class="strut"> </span></p>
  284. <p class="stm run hide_run" id="t284"> <span class="key">def</span> <span class="nam">has_key</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">key</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  285. <p class="pln" id="t285"> <span class="str">"""Return true if a key is in the internal dictionary."""</span><span class="strut"> </span></p>
  286. <p class="stm run hide_run" id="t286"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  287. <p class="stm run hide_run" id="t287"> <span class="key">return</span> <span class="nam">key</span> <span class="key">in</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="strut"> </span></p>
  288. <p class="pln" id="t288"><span class="strut"> </span></p>
  289. <p class="stm run hide_run" id="t289"> <span class="key">def</span> <span class="nam">__eq__</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">other</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  290. <p class="pln" id="t290"> <span class="str">"""Shared memory dictionary equality check with another shared memory dictionary."""</span><span class="strut"> </span></p>
  291. <p class="stm run hide_run" id="t291"> <span class="key">return</span> <span class="nam">isinstance</span><span class="op">(</span><span class="nam">other</span><span class="op">,</span> <span class="nam">SHMDict</span><span class="op">)</span> <span class="key">and</span> <span class="nam">self</span><span class="op">.</span><span class="nam">safe_shm_name</span> <span class="op">==</span> <span class="nam">other</span><span class="op">.</span><span class="nam">safe_shm_name</span><span class="strut"> </span></p>
  292. <p class="pln" id="t292"><span class="strut"> </span></p>
  293. <p class="stm run hide_run" id="t293"> <span class="key">def</span> <span class="nam">__ne__</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">other</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  294. <p class="pln" id="t294"> <span class="str">"""Shared memory dictionary non-equality check with another shared memory dictionary."""</span><span class="strut"> </span></p>
  295. <p class="stm run hide_run" id="t295"> <span class="key">return</span> <span class="key">not</span> <span class="nam">isinstance</span><span class="op">(</span><span class="nam">other</span><span class="op">,</span> <span class="nam">SHMDict</span><span class="op">)</span> <span class="key">or</span> <span class="op">(</span><span class="strut"> </span></p>
  296. <p class="pln" id="t296"> <span class="nam">isinstance</span><span class="op">(</span><span class="nam">other</span><span class="op">,</span> <span class="nam">SHMDict</span><span class="op">)</span> <span class="key">and</span> <span class="nam">self</span><span class="op">.</span><span class="nam">safe_shm_name</span> <span class="op">!=</span> <span class="nam">other</span><span class="op">.</span><span class="nam">safe_shm_name</span><span class="strut"> </span></p>
  297. <p class="pln" id="t297"> <span class="op">)</span><span class="strut"> </span></p>
  298. <p class="pln" id="t298"><span class="strut"> </span></p>
  299. <p class="stm run hide_run" id="t299"> <span class="key">def</span> <span class="nam">__contains__</span><span class="op">(</span><span class="nam">self</span><span class="op">,</span> <span class="nam">key</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  300. <p class="pln" id="t300"> <span class="str">"""Check if a key exists inside the dictionary."""</span><span class="strut"> </span></p>
  301. <p class="stm run hide_run" id="t301"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  302. <p class="stm run hide_run" id="t302"> <span class="key">return</span> <span class="nam">key</span> <span class="key">in</span> <span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="strut"> </span></p>
  303. <p class="pln" id="t303"><span class="strut"> </span></p>
  304. <p class="stm run hide_run" id="t304"> <span class="key">def</span> <span class="nam">__iter__</span><span class="op">(</span><span class="nam">self</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  305. <p class="pln" id="t305"> <span class="str">"""Iterate through the dictionary keys."""</span><span class="strut"> </span></p>
  306. <p class="stm run hide_run" id="t306"> <span class="key">with</span> <span class="nam">self</span><span class="op">.</span><span class="nam">exclusive_lock</span><span class="op">(</span><span class="op">)</span><span class="op">:</span><span class="strut"> </span></p>
  307. <p class="stm run hide_run" id="t307"> <span class="key">return</span> <span class="nam">iter</span><span class="op">(</span><span class="nam">self</span><span class="op">.</span><span class="nam">__internal_dict</span><span class="op">)</span><span class="strut"> </span></p>