Using Django Lock Manager¶

This is the main module file for Django Lock Manager (django-lockmgr) and contains lock management functions/classes.

There are two ways you can use Django Lock Manager:

The first (and recommended) way, is to use the context manager class LockMgr.

The second (lower level) way, is to use the lock functions directly, such as get_lock(), unlock(), and set_lock().

Using the context manager LockMgr (recommended)¶

LockMgr is a wrapper class for the various locking functions in this module, e.g. get_lock(), and is designed to be used as a context manager, i.e. using a with statement.

It’s strongly recommended to use django-lockmgr via the LockMgr context manager unless you have a specific need for manual lock management, as it greatly reduces the risk of “stuck locks” due to human error, or incorrect exception handling.

By using django-lockmgr via this context manager, it ensures you don’t forget to release any locks after you’ve finished with the resources you were using.

Not only that, but it also ensures in the event of an exception, or an unexpected crash of your application, that your locks will usually be safely released by LockMgr.__exit__().

>>> from lockmgr.lockmgr import LockMgr
>>> try:
...     with LockMgr('mylock', 60) as l:
...         print('Doing stuff with mylock locked.')
...         # Obtain an additional lock for 'otherlock' - will use the same expiry as mylock
...         # Since ``ret`` is set to True, it will return a bool instead of raising Lock
...         if l.lock('otherlock', ret=True):
...             print('Now otherlock is locked...')
...             l.unlock('otherlock')
...         else:
...             print('Not doing stuff because otherlock is already locked...')
...         # If you're getting close to your lock's expiry (timeout), you can call '.renew()' to add an extra
...         # 2 minutes to your expiry time. Or manually specify the expiry with 'expires=120'
...         sleep(50)
...         l.renew(expires=30) # Add an extra 30 seconds to the expiration of 'mylock'
... except Locked as e:
...     print('Failed to lock. Reason: ', type(e), str(e))

Using the raw module lock management functions¶

In some cases, it might not be suitable to use context management due to a complex application flow, such as the use of threading / multiprocessing, sharing the locks across other applications, etc.

If you need to, you can access the lower level lock management functions by importing this module, or the individual functions.

Here’s some examples:

First, let’s get a lock using get_lock() that expires in 10 seconds, and wait a few seconds.

>>> from lockmgr import lockmgr
>>> lk = lockmgr.get_lock('my_app:somelock', expires=10)
>>> sleep(5)

Since our lock is going to expire soon, we’ll use renew_lock() to reset the expiration time to 20 seconds from now.

>>> lk = lockmgr.renew_lock(lk, 20)   # Change the expiry time to 20 seconds from now
>>> sleep(15)

Using is_locked(), we can confirm that the lock ``my_app:somelock` is still locked:

>>> lockmgr.is_locked('my_app:somelock') # 15 seconds later, the lock is still locked
True

Finally, we use unlock() to release the lock. You can pass either a string lock name such as my_app:somelock, or you can also pass a Lock database object i.e. the result from get_lock(). Use whichever parameter type you prefer, it doesn’t make a difference.

>>> lockmgr.unlock(lk)

Extra documentation¶

This is not the end of the documentation, this is only the beginning! :)

You’ll find detailed documentation on the pages for each function / class / method. Most things are documented using PyDoc, which means you can view usage information straight from most Python IDEs (e.g. PyCharm and VS Code), as well as via the help() function inside of the Python REPL.

Browsable HTML API docs

We have online documentation for this module, which shows the usage information for each individual function and class method in this module.

Python REPL help

Using the help() function, you can view help on modules, classes, functions and more straight from the REPL:

$ ./manage.py shell
Python 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from lockmgr import lockmgr
>>> help(lockmgr.get_lock)

Below is a screenshot showing the REPL help page for get_lock()

exception lockmgr.lockmgr.LockFail(*args, lock: lockmgr.models.Lock = None)[source]¶: Raised when locks were requested, with failure/rollback if any already existed.

class lockmgr.lockmgr.LockMgr(name, expires: Optional[int] = 600, locked_by=None, lock_process=None, wait: int = None)[source]¶

LockMgr is a wrapper class for the various locking functions in this module, e.g. get_lock(), and is designed to be used as a context manager, i.e. using a with statement.

By using django-lockmgr via this context manager, it ensures you don’t forget to release any locks after you’ve finished with the resources you were using.

Not only that, but it also ensures in the event of an exception, or an unexpected crash of your application, that your locks will usually be safely released by __exit__().

Usage:

Using a with statement, create a LockMgr for mylock with automatic expiration if held for more than 60 seconds. After the with statement is completed, all locks created will be removed.

>>> try:
...     with LockMgr('mylock', 60) as l:
...         print('Doing stuff with mylock locked.')
...         # Obtain an additional lock for 'otherlock' - will use the same expiry as mylock
...         # Since ``ret`` is set to True, it will return a bool instead of raising Lock
...         if l.lock('otherlock', ret=True):
...             print('Now otherlock is locked...')
...             l.unlock('otherlock')
...         else:
...             print('Not doing stuff because otherlock is already locked...')
... except Locked as e:
...     print('Failed to lock. Reason: ', type(e), str(e))

You can also use renew() to request more time / re-create the lock if you’re close to, or have already exceeded the lock expiration time (defaults to 10 mins).

>>> try:
...     with LockMgr('mylock', 60) as l:
...         print('Doing stuff with mylock locked.')
...         sleep(50)
...         l.renew(expires=30)    # Add an additional 30 seconds of time to the lock expiration
...         sleep(50)              # It's now been 100 seconds. 'mylock' should be expired.
...         # We can still renew an expired lock when using LockMgr. It will simply re-create the lock.
...         l.renew()              # Add an additional 120 seconds (default) of time to the lock expiration
... except Locked as e:
...     print('Failed to lock. Reason: ', type(e), str(e))

expires = None¶: The user supplied expiration time in seconds

lock(name, expires: int = None, ret: bool = False, wait: int = None)[source]¶

Obtains a lock using get_lock() and appends it to _locks if successful.

If the argument ret is False (default), it will raise Locked if the lock couldn’t be obtained.

Otherwise, if ret is True, it will simply return False if the requested lock name is already locked.

Parameters

name (str) – A unique name to identify your lock
expires (int) – (Default: 600 sec) How long before this lock is considered stale and forcefully released?
ret (bool) – (Default: False) Return False if locked, instead of raising Locked.
wait (int) – (Optional) Retry obtaining the lock for this many seconds. MUST be divisible by 5. If not empty, will retry obtaining the lock every 5 seconds until wait seconds

Raises

Locked – If the requested lock name is already locked elsewhere, Locked will be raised

Return bool success

True if successful. If ret is true then will also return False on failure.

lock_process = None¶: Usually None, but sometimes may represent the process ID this lock belongs to

locked_by = None¶: Who/what created this lock - usually the hostname unless manually specified

main_lock = None¶: The Lock object created at the start of a with LockManager('xyz') statement

name = None¶: The lock name (from the constructor)

renew(lock: Union[str, lockmgr.models.Lock] = None, expires: int = 120, add_time: bool = True, **kwargs) → lockmgr.models.Lock[source]¶

Add expires seconds to the lock expiry time of lock. If lock isn’t specified, will default to the class instance’s original lock main_lock

Alias for renew_lock() - but with add_time and create set to True by default, instead of False.

With no arguments specified, this method will renew the main lock of the class main_lock for an additional 2 minutes (or if the lock is already expired, will re-create it with 2 min expiry).

Example usage:

>>> with LockMgr('mylock', expires=30) as l:
...     sleep(10)
...     l.renew(expires=60)                  # Add 60 seconds more time to 'mylock' expiration
...     l.main_lock.refresh_from_db()
...     print(l.main_lock.expires_seconds)   # Output: 79
...     l.renew('lockx', expires=60)         # Add 60 seconds more time to 'lockx' expiration

Parameters

lock (Lock) – Name of the lock to renew
lock – A Lock object to renew
expires (int) – (Default: 120) If not add_time, then this is the new expiration time in seconds from now. If add_time, then this many seconds will be added to the expiration time of the lock.
add_time (bool) – (Default: True) If True, then expires seconds will be added to the existing lock expiration time, instead of setting the expiration time to now + expires

Extra Keyword Arguments

Key bool create: (Default: True) If True, then create a new lock if it doesn’t exist / already expired
Key str locked_by: (Default: system hostname) What server/app is trying to obtain this lock?
Key int lock_process: (Optional) The process ID requesting the lock

Exceptions

Raises: LockNotFound – Raised if the requested lock doesn’t exist / is already expired and create is False.
Return Lock lock: The Lock object which was renewed

unlock(lock: Union[lockmgr.models.Lock, str] = None)[source]¶: Alias for unlock()

wait = None¶: How long to wait for a lock before giving up. If this is None then waiting will be disabled

exception lockmgr.lockmgr.LockNotFound[source]¶: Raised when a requested lock doesn’t exist

class lockmgr.lockmgr.LockSetResult(**kwargs)[source]¶

class lockmgr.lockmgr.LockSetStatus(**kwargs)[source]¶

exception lockmgr.lockmgr.Locked[source]¶: Raised when a lock already exists with the given name

lockmgr.lockmgr.clean_locks()[source]¶: Deletes expired Lock objects.

lockmgr.lockmgr.get_lock(name, expires: Optional[int] = 600, locked_by: str = None, lock_process: int = None) → lockmgr.models.Lock[source]¶

READ THIS: It’s best to use LockMgr as it automatically handles locking and unlocking using with.

Calls clean_locks() to remove any expired locks, checks for any existing locks using a FOR UPDATE transaction, then attempts to obtain a lock using the Lock model payments.models.Lock

If name is already locked, then Locked will be raised.

Otherwise, if it was successfully locked, a payments.models.Lock object for the requested lock name will be returned.

Usage:

>>> try:   # Obtain a lock on 'mylock', with an automatic expiry of 60 seconds.
...     mylock = get_lock('mylock', 60)
...     print('Successfully locked mylock')
... except Locked as e:
...     print('Failed to lock. Reason: ', type(e), str(e))
... finally:  # Regardless of whether there was an exception or not, remember to remove the lock!
...     print('Removing lock on "mylock"')
...     unlock(mylock)

Parameters

name (str) – A unique name to identify your lock
expires (int) – (Default: 600 sec) How long before this lock is considered stale and forcefully released? Set this to 0 for a lock which will never expire (must manually call unlock())
locked_by (str) – (Default: system hostname) What server/app is trying to obtain this lock?
lock_process (int) – (Optional) The process ID requesting the lock

Raises

Locked – If the requested lock name is already locked elsewhere, Locked will be raised

Return Lock lock

If successfully locked, will return the payments.models.Lock of the requested lock.

lockmgr.lockmgr.is_locked(name: Union[lockmgr.models.Lock, str]) → bool[source]¶: Cleans expired locks, then returns True if the given lock key name exists, otherwise False

lockmgr.lockmgr.renew_lock(lock: Union[str, lockmgr.models.Lock], expires: int = 600, add_time: bool = False, **kwargs) → lockmgr.models.Lock[source]¶

Renew an existing lock for more expiry time.

Note: This function will NOT reduce a lock’s expiry time, only lengthen. If add_time is False, and the new expiration time expires is shorter than the lock’s existing expiration time, then the lock’s expiry time will be left untouched.

Example - Renew an existing lock:

>>> lk = get_lock('my_app:somelock', expires=10)
>>> sleep(5)
>>> lk = renew_lock(lk, 20)   # Change the expiry time to 20 seconds from now
>>> sleep(15)
>>> is_locked('my_app:somelock') # 15 seconds later, the lock is still locked
True

Example - Try to renew, but get a new lock if it’s already been released:

>>> lk = get_lock('my_app:somelock', expires=5)
>>> sleep(10)
>>> lk = renew_lock(lk, 20, create=True)   # If the lock is expired/non-existant, make a new lock
>>> sleep(15)
>>> is_locked('my_app:somelock') # 15 seconds later, the lock is still locked
True

Parameters

lock (Lock) – Name of the lock to renew
lock – A Lock object to renew
expires (int) – (Default: 600) If not add_time, then this is the new expiration time in seconds from now. If add_time, then this many seconds will be added to the expiration time of the lock.
add_time (bool) – (Default: False) If True, then expires seconds will be added to the existing lock expiration time, instead of setting the expiration time to now + expires

Key bool create

(Default: False) If True, then create a new lock if it doesn’t exist / already expired.

Key str locked_by

(Default: system hostname) What server/app is trying to obtain this lock?

Key int lock_process

(Optional) The process ID requesting the lock

Raises

LockNotFound – Raised if the requested lock doesn’t exist / is already expired and create is False.

Return Lock lock

The Lock object which was renewed

lockmgr.lockmgr.set_lock(*locks, timeout=600, fail=False, renew=True, create=True, **options) → lockmgr.lockmgr.LockSetResult[source]¶

This function is for advanced users, offering multiple lock creation, renewing, along with “all or nothing” locking with database rollback via the argument fail.

Unlike other lock management functions, set_lock returns a LockSetResult object, which is designed to allow you to see clearly as to what locks were created, renewed, or skipped.

Example Usage

Let’s set two locks, hello and world.

>>> res = set_lock('hello', 'world')
>>> res['locks']
[<Lock name='hello' locked_by='example.org' locked_until='2019-11-22 02:01:55.439390+00:00'>,
 <Lock name='world' locked_by='example.org' locked_until='2019-11-22 02:01:55.442734+00:00'>]
>>> res['counts']
{'created': 2, 'renewed': 0, 'skip_create': 0, 'skip_renew': 0}

If we run set_lock again with the same arguments, we’ll still get the locks list, but we’ll see the counts show that they were renewed instead of created.

>>> x = set_lock('hello', 'world')
>>> x['locks']
[<Lock name='hello' locked_by='example.org' locked_until='2019-11-22 02:03:06.762620+00:00'>,
 <Lock name='world' locked_by='example.org' locked_until='2019-11-22 02:03:06.766804+00:00'>]
>>> x['counts']
{'created': 0, 'renewed': 2, 'skip_create': 0, 'skip_renew': 0}

Since the result is an object, you can also access attributes via dot notation, as well as dict-like notation.

We can see inside of the statuses list - the action that was taken on each lock we specified, so we can see what locks were created, renewed, or skipped etc.

>>> x.statuses[0]
('hello', {'was_locked': True, 'status': 'extend', 'locked': True})
>>> x.statuses[1]
('world', {'was_locked': True, 'status': 'extend', 'locked': True})

Parameters

locks (str) – One or more lock names, as positional arguments, to create or renew.
timeout (int) – On existing locks, update locked_until to now + timeout (seconds)
fail (bool) – (Default: False) If True, all lock creations will be rolled back if an existing lock is encountered, and LockFail will be raised.
renew (bool) – (Default: True) If True, any existing locks in locks will be renewed to now + timeout (seconds). If False, existing locks will just be skipped.
create (bool) – (Default: True) If True, any names in locks which aren’t yet locked, will have a lock created for them, with their expiry set to timeout seconds from now.

Key str locked_by

(Default: system hostname) What server/app is trying to obtain this lock?

Key int process_id

(Optional) The process ID requesting the lock

Return LockSetResult results

A LockSetResult object containing the results of the set_lock operation.

lockmgr.lockmgr.unlock(lock: Union[lockmgr.models.Lock, str])[source]¶

Releases a given lock - either specified as a string name, or as a payments.models.Lock object.

Usage:

>>> mylock = get_lock('mylock', expires=60)
>>> unlock('mylock') # Delete the lock by name
>>> unlock(mylock)   # Or by Lock object.

Parameters

lock (Lock) – The name of the lock to release
lock – A Lock object to release

API Docs (lockmgr.lockmgr)¶

Functions

clean_locks()

Deletes expired Lock objects.

get_lock(name[, expires, locked_by, …])

READ THIS: It’s best to use LockMgr as it automatically handles locking and unlocking using with.

is_locked(name)

Cleans expired locks, then returns True if the given lock key name exists, otherwise False

renew_lock(lock[, expires, add_time])

Renew an existing lock for more expiry time.

set_lock(*locks[, timeout, fail, renew, create])

This function is for advanced users, offering multiple lock creation, renewing, along with “all or nothing” locking with database rollback via the argument fail.

unlock(lock)

Releases a given lock - either specified as a string name, or as a payments.models.Lock object.

Classes

LockMgr(name[, expires, locked_by, …])

LockMgr is a wrapper class for the various locking functions in this module, e.g.

Exceptions

LockNotFound

Raised when a requested lock doesn’t exist

Locked

Raised when a lock already exists with the given name