Pure Python tools for reading and writing all TIFF IFDs, sub-IFDs, and tags.

Overview

Tiff Tools Build Status codecov.io License

Pure Python tools for reading and writing all TIFF IFDs, sub-IFDs, and tags.

Developed by Kitware, Inc. with funding from The National Cancer Institute.

Example

import tifftools
info = tifftools.read_tiff('photograph.tif')
info['ifds'][0]['tags'][tifftools.Tag.ImageDescription.value] = {
    'data': 'A dog digging.',
    'datatype': tifftools.Datatype.ASCII
}
exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0]
exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
    'data': [54, 10],
    'datatype': tifftools.Datatype.RATIONAL
}
tifftools.write_tiff(info, 'photograph_tagged.tif')

Commands

tifftools --help and tifftools --help provide usage details.

  • tifftools split [--subifds] [--overwrite] source [prefix]: split a tiff file into separate files. This is also available as the library function tifftools.tiff_split.
  • tifftools concat [--overwrite] source [source ...] output: merge multiple tiff files together. Alias: tifftools merge. This is also available as the library function tifftools.tiff_concat.
  • tifftools dump [--max MAX] [--json] source [source ...]: print information about a tiff file, including all tags, IFDs, and subIFDs. Alias: tifftool info. This is also available as the library function tifftools.tiff_dump.
  • tifftools set source [--overwrite] [output] [--set TAG[:DATATYPE][, ] VALUE] [--unset TAG:[, ]] [--setfrom TAG[, ] TIFFPATH] : modify, add, or remove tags. This is also available as the library function tifftools.tiff_set.

Library Functions

  • read_tiff
  • write_tiff
  • Constants
  • Tag
  • Datatype
  • get_or_create_tag
  • EXIFTag, GPSTag, etc.

Installation

tifftools is available on PyPI and conda-forge.

To install with pip from PyPI:

pip install tifftools

To install with conda:

conda install -c conda-forge tifftools

Purpose

tifftools provides a library and a command line program for maniplulating TIFF files. It can split multiple images apart, merge images together, set any tag in any IFD, and dump all IFDs and tags in a single command. It only uses python standard library modules, and is therefore widely compatible.

Rationale

There was a need to combine images from multiple TIFF files without altering the image data or losing any tag information. Further, when changing tag values, it was essential that the old values were fully removed from the output.

The command line tools associated with libtiff are commonly used for similar purposes. The libtiff command tools have significant limitations: tiffdump and tiffinfo require multiple commands to see information from all IFDs. tiffset does not remove data from a file; rather it appends to the file to only reference new data, leaving the old values inside the file. tiffsplit doesn't keep tags it doesn't recognize, loosing data. tiffcp always reencodes images and will fail for compression types it does not know.

Likewise, there is a wide variety of EXIF tools. For the most part, these only alter tags, usually by appending to the existing file. ImageMagick's convert command also recompresses images as it combines them.

Many programs deal with both classic and BigTIFF. Some will start writing a classic TIFF, but leave a small amount of unused space just after the file header. If the file exceeds 4Gb, parts of the file are rewritten to convert it to a BigTIFF file, leaving small amounts of abandoned data within the file.

tifftools fills this need. All tags are copied, even if unknown. Files are always rewritten so that there is never abandoned data inside the file. tifftools dump shows information on all IFDs and tags. Many of the command line options are directly inspired from libtiff.

tifftools does NOT compress or decompress any image data. This is not an image viewer. If you need to recompress an image or otherwise manipulate pixel data, use libtiff or another library.

As an explicit example, with libtiff's tiffset, tag data just gets dereferenced and is still in the file:

$ grep 'secret' photograph.tif  || echo 'not present'
not present
$ tiffset -s ImageDescription "secret phrase" photograph.tif
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: secret phrase
$ grep 'secret' photograph.tif  || echo 'not present'
Binary file photograph.tif matches
$ tiffset photograph.tif -s ImageDescription "public phrase"
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: public phrase
$ grep 'secret' photograph.tif  || echo 'not present'
Binary file photograph.tif matches

Whereas, with tifftools:

$ grep 'secret' photograph.tif || echo 'not present'
not present
$ tifftools set -y -s ImageDescription "secret phrase" photograph.tif
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: secret phrase
$ grep 'secret' photograph.tif || echo 'not present'
Binary file photograph.tif matches
$ tifftools set -y photograph.tif -s ImageDescription "public phrase"
$ tiffinfo photograph.tif | grep ImageDescription
  ImageDescription: public phrase $ grep 'secret' photograph.tif || echo
  'not present' not present

TIFF File Structure

TIFF Files consist of one or more IFDs (Image File Directories). These can be located anywhere within the file, and are referenced by their absolute position within the file. IFDs can refer to image data; they can also contain a collection of metadata (for instance, EXIF or GPS data). Small data values are stored directly in the IFD. Bigger data values (such as image data, longer strings, or lists of numbers) are referenced by the IFD and are stored elsewhere in the file.

In the simple case, a TIFF file may have a list of IFDs, each one referencing the next. However, a complex TIFF file, such as those used by some Whole-Slide Image (WSI) microscopy systems, can have IFDs organized in a branching structure, where some IFDs are in a list and some reference SubIFDs with additional images.

TIFF files can have their primary data stored in either little-endian or big-endian format. Offsets to data are stored as absolute numbers inside a TIFF file. There are two variations: "classic" and "BigTIFF" which use 32-bits and 64-bits for these offsets, respectively. If the file size exceeds 4 Gb or uses 64-bit integer datatypes, it must be written as a BigTIFF.

Limitations

Unknown tags that are offsets and have a datatype other than IFD or IFD8 won't be copied properly, as it is impossible to distinguish integer data from offsets given LONG or LONG8 datatypes. This can be remedied by defining a new TiffConstant record which contains a bytecounts entry to instruct whether the offsets refer to fixed length data or should get the length of data from another tag.

Because files are ALWAYS rewritten, tifftools is slower than libtiff's tiffset and most EXIF tools.

Comments
  • TIFF IFD concatenations/removals output images that fail JHOVE validation check for value offset word-alignment

    TIFF IFD concatenations/removals output images that fail JHOVE validation check for value offset word-alignment

    (https://jhove.openpreservation.org/modules/tiff/)

    > python -c 'import tifftools;tifftools.tiff_concat(["good1.svs", "good2.svs"], "out.svs", overwrite=True)'
    > jhove -m TIFF-hul out.svs
    
    Jhove (Rel. 1.24.1, 2020-03-16)
     Date: 2021-07-15 20:14:30 MDT
     RepresentationInformation: out.svs
      ReportingModule: TIFF-hul, Rel. 1.9.2 (2019-12-10)
      LastModified: 2021-07-15 20:14:20 MDT
      Size: 875685820
      Format: TIFF
      Status: Not well-formed
      SignatureMatches:
       TIFF-hul
      ErrorMessage: Value offset not word-aligned: 8842289
       ID: TIFF-HUL-4
       Offset: 8858224
      MIMEtype: image/tiff
    

    https://web.archive.org/web/20160324105748/https://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf on page 15 (about IFD entries) says:

    Bytes 8-11 The Value Offset, the file offset (in bytes) of the Value for the field. The Value is expected to begin on a word boundary; the correspond- ing Value Offset will thus be an even number. This file offset may point anywhere in the file, even after the image data.

    opened by fiendish 4
  • Appending a new tag to an existing TIFF image

    Appending a new tag to an existing TIFF image

    Hi,

    I am trying to add a new tag to a TIF file using Python3.

    I have worked through your code and I am now able to understand the TIFF structure.

    However, I need to add a a new Tag. My code is an attempt to adapt your example for a similar action from the command line:

    info = tifftools.read_tiff(Fpath) info[info, 'ifds'][0]['tags'][tifftools.Tag.ImageDescription.value] = { 'data': 'A dog digging.', 'datatype': tifftools.Datatype.ASCII }

    tifftools.write_tiff(Fpath, OutPath, info)

    I am not the strongest Python programmer, but I am baffled why I cannot update the TIFF TAG dict structure. Can you guide me please?

    regards

    Phil

    opened by pfculverhouse 3
  • Readme example seems to be incorrect

    Readme example seems to be incorrect

    Running the sample code from the readme file gives:

    $ python3 sample.py
    Traceback (most recent call last):
      File "sample2.py", line 8, in <module>
        exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
    TypeError: list indices must be integers or slices, not str
    

    it seems that we are missing level for exifs (SubIDF?).

    The following does works:

    $ diff -u sample.py.orig sample.py
    --- sample.py.orig      2022-02-28 18:19:35.000000000 +0100
    +++ sample.py   2022-02-28 18:19:52.000000000 +0100
    @@ -4,7 +4,7 @@
         'data': 'A dog digging.',
         'datatype': tifftools.Datatype.ASCII
     }
    -exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0]
    +exififd = info['ifds'][0]['tags'][tifftools.Tag.EXIFIFD.value]['ifds'][0][0]
     exififd['tags'][tifftools.constants.EXIFTag.FNumber.value] = {
         'data': [54, 10],
         'datatype': tifftools.Datatype.RATIONAL
    $ python3 sample.py
    $ tifftools dump photograph_tagged.tif | grep FNumber
          FNumber 33437 (0x829D) RATIONAL: 54 10 (5.4)
    

    Side note: the above code will fail if the TIFF file doesn't already contains EXIFs. Is the following snippet the correct way to add EXIF IFD?

    try:
        exif = info["ifds"][0]["tags"][tifftools.Tag.EXIFIFD.value]
    except KeyError:
        exif = {
            "datatype": tifftools.Datatype.IFD,
            "ifds": [[{"tags": {}, "path_or_fobj": info["ifds"][0]["path_or_fobj"]}]],
        }
        info["ifds"][0]["tags"][tifftools.Tag.EXIFIFD.value] = exif
    exififd = exif["ifds"][0][0]
    # add tags herunder
    
    opened by AmedeeBulle 2
  • Examples of write tiffs from scratch, not inheriting info from an existing tiff

    Examples of write tiffs from scratch, not inheriting info from an existing tiff

    Please could you provide more examples for how to use this tool to write tiffs from scratch? Thanks SO much for providing this tool. I really hope I can make it work for me!

    Problem:

    • I would like to write out numpy arrays to bigtiff with custom metadata AND exif data
    • I'm getting exif data from a jpeg, then generating outputs from that jpeg within a script, that I want to write those outputs to a tiff with exif and metadata

    Alternatives tried:

    • I cant use tifffile because it doesnt deal with exif data
    • I cant use PIL because it doesnt write exif data
    • Rasterio doesnt seem to have an exif option
    • I cant follow the provided example because I dont have a tiff to read as a starting point

    So I'm really hoping I can use tifftools, but I cant figure out (from the one provided example) how to write a tiff entirely from scratch using arrays

    Question: The provided example is useful, but only deals with the case where all the info is obtained from an existing file

    info = tifftools.read_tiff(file)

    But how would you create that info object from an numpy array, metadata, and exif separately? where metadata and exif are two separate dicts or JSON objects (or whatever)?

    I also found this that helpfully explains how to add info to an existing tiff

    In summary, I'm looking for advice for how to construct a viable info object that can be passed to .write_tif like this

    tifftools.write_tif(info, ...)

    where info is constructed from 1) a numpy array containing the image, 2) an array or dictionary of exif data, and 3) an array or dictionary of other metadata (tags)

    opened by dbuscombe-usgs 2
  • Make it easier to add new ifds by not requiring path_or_fobj.

    Make it easier to add new ifds by not requiring path_or_fobj.

    The path_or_fobj internal value is only required for ifds if they need to transfer data from an existing tiff file. If tags are entirely self-contained, this is no longer required to be set.

    opened by manthey 0
  • Better handle NDPI files

    Better handle NDPI files

    NDPI files aren't quite valid tiff files. Rather, they are marked as non-bigtiff, but use 64-bit values for ifd offsets and have some implied upper bits for some data offsets. If a file is larger than 4 Gb or of unknown length, and a NDPI-specific tag is encountered that could have otherwise invalid offsets, read additional IFD offsets as 64-bit values and adjust data offset values according to the NDPI methods as illustrated by the OpenSlide library.

    opened by manthey 0
  • Output values on word boundaries.

    Output values on word boundaries.

    Better handle saving to small tiff. Before, once written to a bigtiff, it was unlikely to convert back to a small tiff since some fields written as LONG8 didn't automatically convert to LONG.

    opened by manthey 0
  • More often generate small tiff

    More often generate small tiff

    Better handle saving to small tiff. Before, once written to a bigtiff, it was unlikely to convert back to a small tiff since some fields written as LONG8 didn't automatically convert to LONG.

    opened by manthey 0
Releases(v1.3.6)
Owner
Digital Slide Archive
Tools for the management, visualization, and analysis of digital pathology data.
Digital Slide Archive
Python virtual filesystem for SQLite to read from and write to S3

Python virtual filesystem for SQLite to read from and write to S3

Department for International Trade 70 Jan 04, 2023
RMfuse provides access to your reMarkable Cloud files in the form of a FUSE filesystem

RMfuse provides access to your reMarkable Cloud files in the form of a FUSE filesystem. These files are exposed either in their original format, or as PDF files that contain your annotations. This le

Robert Schroll 82 Nov 24, 2022
MHS2 Save file editing tools. Transfers save files between players, switch and pc version, encrypts and decrypts.

SaveTools MHS2 Save file editing tools. Transfers save files between players, switch and pc version, encrypts and decrypts. Credits Written by Asteris

31 Nov 17, 2022
Quick and dirty FAT12 filesystem to ZIP file converter

Quick and Dirty FAT12 Filesystem Converter This is a really crappy Python script I wrote to convert a semi-compatible FAT12 filesystem from my HP150's

Tube Time 2 Feb 12, 2022
Extract longest transcript or longest CDS transcript from GTF annotation file or gencode transcripts fasta file.

Extract longest transcript or longest CDS transcript from GTF annotation file or gencode transcripts fasta file.

laojunjun 13 Nov 23, 2022
Nintendo Game Boy music assembly files parser into musicxml format

GBMusicParser Nintendo Game Boy music assembly files parser into musicxml format This python code will get an file.asm from the disassembly of a Game

1 Dec 11, 2021
A simple file sharing tool written in python

Share it A simple file sharing tool written in python Installation If you are using Windows os you can directly Run .exe file -- download If you are

Sachit Yadav 7 Dec 16, 2022
Extract the windows major and minor build numbers from an ISO file, and automatically sort the iso files.

WindowsBuildFromISO Extract the windows major and minor build numbers from an ISO file, and automatically sort the iso files. Features Parse multiple

Podalirius 9 Nov 09, 2022
A Certificate renaming tool made for IEEE CS SBC, SJCE.

PDF Batch Renamer Made for IEEE CS SBC, SJCE How to use? Before using the python script, ensure that pytesseract, pdf2image, opencv and other supporti

Ashwin Kumar U 2 Nov 14, 2021
PaddingZip - a tool that you can craft a zip file that contains the padding characters between the file content.

PaddingZip - a tool that you can craft a zip file that contains the padding characters between the file content.

phithon 53 Nov 07, 2022
CredSweeper is a tool to detect credentials in any directories or files.

CredSweeper is a tool to detect credentials in any directories or files. CredSweeper could help users to detect unwanted exposure of credentials (such as personal information, token, passwords, api k

Samsung 54 Dec 13, 2022
Dragon Age: Origins toolset to extract/build .erf files, patch language-specific .dlg files, and view the contents of files in the ERF or GFF format

DAOTools This is a set of tools for Dragon Age: Origins modding. It can patch the text lines of .dlg files, extract and build an .erf file, and view t

8 Dec 06, 2022
Pure Python tools for reading and writing all TIFF IFDs, sub-IFDs, and tags.

Tiff Tools Pure Python tools for reading and writing all TIFF IFDs, sub-IFDs, and tags. Developed by Kitware, Inc. with funding from The National Canc

Digital Slide Archive 32 Dec 14, 2022
Python function to stream unzip all the files in a ZIP archive: without loading the entire ZIP file or any of its files into memory at once

Python function to stream unzip all the files in a ZIP archive: without loading the entire ZIP file or any of its files into memory at once

Department for International Trade 206 Jan 02, 2023
A tiny Python library for writing multi-channel TIFF stacks.

xtiff A tiny Python library for writing multi-channel TIFF stacks. The aim of this library is to provide an easy way to write multi-channel image stac

23 Dec 27, 2022
CleverCSV is a Python package for handling messy CSV files.

CleverCSV is a Python package for handling messy CSV files. It provides a drop-in replacement for the builtin CSV module with improved dialect detection, and comes with a handy command line applicati

The Alan Turing Institute 1k Dec 19, 2022
QSynthesis is a Python3 API to perform I/O based program synthesis of bitvector expressions.

QSynthesis is a Python3 API to perform I/O based program synthesis of bitvector expressions. It aims at facilitating code deobfuscation. The algorithm is greybox approach combining both a blackbox I/

Quarkslab 103 Dec 30, 2022
A tool written in python to generate basic repo files from github

A tool written in python to generate basic repo files from github

Riley 7 Dec 02, 2021
Python codes for the server and client end that facilitates file transfers. (Using AWS EC2 instance as the server)

Server-and-Client-File-Transfer Python codes for the server and client end that facilitates file transfers. I will be using an AWS EC2 instance as the

Amal Farhad Shaji 2 Oct 13, 2021
Remove [x]_ from StudIP zip Archives and archive_filelist.csv completely

This tool removes the "[x]_" at the beginning of StudIP zip Archives. It also deletes the "archive_filelist.csv" file

Kelke vl 1 Jan 19, 2022