Create a temporary file, take care of removing it if needed

Overview

The with-tempfile library is designed to help handle files that should be modified atomically, so that there is no chance that at any time, another program reading the file would get its incomplete contents, or that an error halfway through writing the updated contents will leave the file incomplete on disk.

It currently provides two functions, write_text and append_text.

Binary file handing is planned.

A command-line tool to overwrite or append to a file atomically is planned.

API

Update or create files atomically

write_text

The write_text function creates a temporary file in the same directory as the target (so that an atomic rename will succeed), writes out the supplied contents, and then atomically renames the temporary file to the target one.

LOCAL_CONFIG: Final = pathlib.Path("/etc/sample/config.toml")

with_tempfile.write_text(LOCAL_CONFIG, toml.dumps(cfg))

append_text

The append_text function creates a temporary file in the same directory as the target (so that an atomic rename will succeed), copies the current contents of the target file if it exists, appends the supplied text, and then atomically renames the temporary file to the target one.

ENC_SINGLE: Final = "ISO-8859-15"
"""A fallback single-byte encoding that has all 256 codepoints as valid."""

ENCODINGS: Final = ("UTF-8", ENC_SINGLE)
"""The names of the encodings to try to decode a file's contents with."""

MARKER_TEXT: Final = f"""
# Autogenerated by sampletool; modifications may be lost!
"""

if CFG_SECTION_NAME not in cfg:
    new_section: Final = f"{MARKER_TEXT}{toml.dumps({CFG_SECTION_NAME: cfg_section})}"
    with_tempfile.append_text(LOCAL_CONFIG, new_section, encoding=ENCODINGS)

Note that this function may be dropped in a future version of the with-tempfile library, since in most cases the desired contents of the new file can be fully constructed and write_text may be used instead.

Path manipulation helpers

The with_tempfile.pathutil module provides the PathLike union type that also includes str, and the as_path function that will convert an object into a pathlike.Path object if it is not already one.

def create_default(path: pathutil.PathLike) -> None:
    """Write the default text to the specified file."""
    with_tempfile.write_text(pathutil.as_path(path), DEFAULT_CONTENTS)

The read_text function is similar to the Path.read_text method from the Python standard pathlib library, but it allows more than one encoding to be tried in order. This may be useful for files that may or may not be valid UTF-8 and a reader that might want to fall back to a single-byte encoding.

contents, encoding = pathutil.read_text(path, encoding=ENCODINGS)
with_tempfile.write_text(path, rot13(contents), encoding=encoding)

Temporary file handling

The with_tempfile.temputil module provides the NamedTemporaryTextFile function that returns a TemporaryTextFile object. It is quite similar to NamedTemporaryFile in Python's standard tempfile library, but the object has the new path and resolved_path properties: both pathllib.Path objects. It also has an unset_delete method that prevents the object destructor from trying to delete the temporary file if the program has already e.g. renamed it.

with temputil.NamedTemporaryTextFile(prefix="info-", suffix=".txt", encoding="UTF-8") as tempf:
    print("this is a test", file=tempf, flush=True)
    tempf.path.rename_to("/etc/sample/info.txt")
    tempf.unset_delete()

There is also a TemporaryDirectory class that is similar to the one in the tempfile library, but also has the path and resolved_path properties. An important difference with the system's TemporaryDirectory class is that the __enter__() method returns a pathlib.Path object instead of a string.

with temputil.TemporaryDirectory(prefix="test-") as tempd:
    tempcfg: Final = tempd / "config.txt"
    ...

Maildir-style directory handling

The MaildirSpool dataclass provides a safecat method that creates a file in the specified temporary directory, writes the provided text to it, and then renames it to a guaranteed unique name in the specified spool directory. This is an implementation of Prof. D. J. Bernstein's Maildir concept, although the naming of the files is left to the caller, they do not necessarily follow the original Maildir specification.

base: Final = pathlib.Path("/path/to/maildir")
maildir: Final = temputil.MaildirSpool(tmpdir=base / "tmp", spooldir=base / "new")
new_file: Final = maildir.safecat(
    prefix="616.42.",
    suffix=".txt",
    contents="This is a test\n",
    encoding="UTF-8",
)
assert new_file.parent == base / "new"

There is also a from_maildir() constructor that uses the tmp and new subdirectory names:

base: Final = pathlib.Path("/path/to/maildir")
maildir: Final = temputil.MaildirSpool.from_maildir(base)
new_file: Final = maildir.safecat(
    prefix="616.42.",
    suffix=".txt",
    contents="This is a test\n",
    encoding="UTF-8",
)
assert new_file.parent == base / "new"

The safecat() top-level function provides a shortcut, specifying all of the parameters at once.

The create_file() method allows a temporary file to be created, then written to over time (not all at once), and then renamed into the spool directory:

base: Final = pathlib.Path("/path/to/maildir")
maildir: Final = temputil.MaildirSpool.from_maildir(base)

tempf = maildir.create_file(
    prefix="616.42.",
    suffix=".txt",
    encoding="UTF-8",
)
print("This is a test", file=tempf)

time.sleep(1)
print("This is only a test", file=tempf)

new_file = tempf.rename_maildir_file()
tempf = None

assert new_file.parent == base / "new"

Contact

The with-tempfile library was written by Peter Pentchev. It is developed in a GitLab repository. This documentation is hosted at Ringlet with a copy at ReadTheDocs.