According to the documentation of os.open the former mode was in fact the flags for the open operation. I renamed this from open_mode to open_flags and added the permission as 0o660 so that the created lockfile now has -rw-rw---- and not -rwxr-xr-x as before.

Since the last update, I am getting the following warnings. I suspect this has to do with the fact that pylint cannot deduce that the correct instance will not be superclassed by ABC (if-else definition of classes).

import filelock

with filelock.FileLock("abc.lock"):
    print("hi")

pylint bla.py 
************* Module bla
bla.py:1:0: C0114: Missing module docstring (missing-module-docstring)
bla.py:3:5: E0110: Abstract class 'WindowsFileLock' with abstract methods instantiated (abstract-class-instantiated)
bla.py:3:5: E0110: Abstract class 'UnixFileLock' with abstract methods instantiated (abstract-class-instantiated)

Description

If the OS cannot open a file because the path is bad, it does not make sense to repeatedly attempt to open the file as it will continue to fail, indefinitely if the -1 default timeout is used, without any feedback to the user.

This modifies the behavior to raise the OSError/IOError exception received on Windows or Unix so FileLock exits rather than a futile infinite loop that will never succeed.

Tests are written to demonstrate the behavior.

I thought the convention was that dropping support for a Python version was a breaking change, so this should have been 4.0.0? Maybe there are other philosophies.

Hi there, When I run the following script, it runs successfully on Linux but fails on Windows 10:

import filelock

lock = filelock.FileLock('test.txt')
with lock:
    f = open('test.txt', 'w')
    f.write('hello')
    f.close()

The error on Windows is:

PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "lock.py", line 9, in <module>
    f.close()
PermissionError: [Errno 13] Permission denied

However, if I move f.close() outside of the context manager, the script successfully runs on Windows and Linux:

lock = filelock.FileLock('test.txt')
with lock:
    f = open('test.txt', 'w')
    f.write('hello')
f.close()

Why exactly must the f.close() be placed outside the context manager on Windows, but not on Linux? Is it related to msvcrt.locking()?

I'm using filelock 2.0.7, Windows 10 (x64) and Debian Jessie (x64).

Hi, found this from a search. Please add the 4 line usage. I expect something like this:

LOCK_FILE = ...
LOCK = InterprocessLock(LOCK_FILE)

def foo():
    with LOCK:
        do_something()

Thanks

Fix misspelled keyword argument poll_interval for method acquire. Fixes #62.

I tried my best to add the backward compatibility layer in this PR to not break existing codebases.

On python 3.9 with filelock 3.8.0, this code hangs:

from multiprocessing import Pool
from filelock import FileLock


def run(i):
    print(f"got the lock in multi process [{i}]")


with FileLock("tmp.lock"):
    with Pool(2) as pool:
        pool.map(run, range(2))

This is because the subprocesses tries to acquire the lock from the main process for some reason. This is not the case in older versions of python.

This can cause many issues in python 3.9.

For example, we use multiprocessing to run a pool of jobs and we use filelock to prevent running the same pool of job several times and avoid writing collisions.

First reported in https://github.com/huggingface/datasets/issues/4113

By default this package will always print to a logger, but for my use case it only ends up cluttering the logs with "released, acquired, released, acquired, etc". I'd appreciate it if you could add the option to disable the logger :)

This is a small fix for people who employ pathlib.Path for their path handling. It allows a Path object to be passed to the constructor of FileLock. It is then converted to a normal str when storing it inside the object leaving the rest of the library unaffected.

We are using py-filelock 3.0.12 and have experienced a problem when multiple threads are trying to acquire the lock on the same file at almost the same time. This happens quite often in our AWS EC2 instance and causes data corruption. We are just following the examples in the documentation to use the module. Is there any way to avoid this?

I have been intermittently seeing this error (was thinking possibly due to a race condition), but one of my coworkers started running into this issue repeatedly. Any idea why this could be happening? Reproducing solely on macOS

Code I'm running:

lock_path = '/tmp/locks/my_lock.lock'
with FileLock(lock_path, timeout=0):
   ... 

I see that in _acquire() the os.O_CREAT flag is being provided to os.open(), so shouldn't the file be created if it does not exist?

Sanitized traceback:

Traceback (most recent call last):
  File "/Users/my_user/.virtualenvs/my_proj/my_proj/cache.py", line 311, in update_cache
    with FileLock(lock_path, timeout=0):
  File "/Users/my_user/.virtualenvs/my_env/lib/python3.10/site-packages/filelock/_api.py", line 220, in __enter__
    self.acquire()
  File "/Users/my_user/.virtualenvs/my_env/lib/python3.10/site-packages/filelock/_api.py", line 173, in acquire
    self._acquire()
  File "/Users/my_user/.virtualenvs/my_env/lib/python3.10/site-packages/filelock/_unix.py", line 35, in _acquire
    fd = os.open(self._lock_file, open_mode)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/locks/my_lock.lock'

I ran into a strange bug when trying to lock a file on a network file-system mounted inside a container, where the lock file was created but for some reason it seems as though the file-handle wasn't properly returned. My process then got stuck waiting for the lock to be released (when it had in fact created the lock). Looking at the following code, it seems that if the OSError errno isn't EEXIST, ENOENT or EACCES, then it is assumed the file is locked

https://github.com/tox-dev/py-filelock/blob/4730a40b87cc4b094330b2af7723658428323d60/src/filelock/_soft.py#L23-L32

wouldn't it be more robust to do something like

 try: 
     fd = os.open(self._lock_file, mode) 
 except OSError as exception: 
     if (exception.errno == EEXIST # expected if cannot lock 
             or (if exception.errno == EACCES and sys.platform == "win32"):  # note windows does not allow you to make a folder r/o only files 
         pass 
     else:
         raise

Or do you actually want the code to keep attempting to try creating the lock on other OSErrors?

The API documentation of filelock.FileLock simply reads:

alias of filelock._unix.UnixFileLock

Naturally, this is only true on platforms supporting fcntl.flock, else it might be a WindowsFileLock or SoftFileLock. I assume that ReadTheDocs runs and produces the HTML pages on a Linux system.

I would expect the docs to instead indicate that this is an alias for the lock implementation specific to the platform the code is being run on, which may be any of the three classes. Ideally, this would be true also for filelock.FileLock.__doc__ at runtime (e.g. for help() in the REPL).

I'm not very familiar with Sphinx, so I don't know what the best approach for this is. My intuitive attempt would be to make FileLock a subclass of the relevant implementation (i.e. class FileLock(_FileLock) in src/filelock/__init__.py) and give it its own docstring. However, the 'Bases' line in the Sphinx-generated docs would still have to be fixed or suppressed for this particular class.

The creation of a lockfile with …

lock = FileLock("/var/lock/foo.lock")

… leads to these file permissions: -rwxr-xr-x

Is there any way to prevent that the lock file becomes an executable with root ownership?

(Version: 3.0.12-2 in Ubuntu 20.04)

Hi,

On Windows 10.0.19041.687 Pro x64. Python 3.7.0: x64.

Here is a test script:

import tempfile
import pathlib
import threading
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from filelock import FileLock
import time

TEST_FILE = pathlib.Path(tempfile.gettempdir()) / 'test_file.txt'
TEST_LOCK_FILE =  pathlib.Path(tempfile.gettempdir()) / 'test_file.txt.lock'
LOCK = FileLock(TEST_LOCK_FILE)

def test():
    with LOCK:
        assert TEST_FILE.read_text() == 'hi'
        TEST_FILE.write_text('')
        assert TEST_FILE.read_text() == ''
        TEST_FILE.write_text('hi')
        assert TEST_FILE.read_text() == 'hi'
        return True

if __name__ == '__main__':
    print(f"Test file: {TEST_FILE}")
    print(f"Test lock file: {TEST_LOCK_FILE}")
    TEST_FILE.write_text('hi')

    results = []

    # works with ProcessPoolExecutor but not with ThreadPoolExecutor
    # It also is more likely to work if we sleep after calling submit()
    with ThreadPoolExecutor(max_workers=16) as pool:
        for i in range(100):
            if i % 10 == 0:
                print (f"Loop: {i+1}")
            results.append(pool.submit(test))

        for idx, result in enumerate(results):
            print (f"Checking result: {idx + 1}")
            assert result.result() == True

Example run:

PS C:\Users\csm10495\Desktop> python .\file_lock_test.py
Test file: C:\Users\csm10495\AppData\Local\Temp\test_file.txt
Test lock file: C:\Users\csm10495\AppData\Local\Temp\test_file.txt.lock
Loop: 1
Loop: 11
Loop: 21
Loop: 31
Loop: 41
Loop: 51
Loop: 61
Loop: 71
Loop: 81
Loop: 91
Checking result: 1
Traceback (most recent call last):
  File ".\file_lock_test.py", line 38, in <module>
    assert result.result() == True
  File "C:\Python37\lib\concurrent\futures\_base.py", line 425, in result
    return self.__get_result()
  File "C:\Python37\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "C:\Python37\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File ".\file_lock_test.py", line 18, in test
    assert TEST_FILE.read_text() == 'hi'
AssertionError

Using a ThreadPoolExecutor seems to lead to assertion errors making me think the lock file wasn't atomically created. If I sleep a bit (like .01), after doing submit() it seems to work, but sort of defeats the purpose of the test.