An open-source application for biological image analysis

Hello - I am trying to install the CP developer's version as outlined on https://github.com/CellProfiler/CellProfiler/wiki/Cellprofiler-developer%27s-version-installation-for-Mac, under "Methods that work". However, I get an error close to the end of installing dependencies with Homebrew:

Error: Failed executing: /bin/sh ./setup.sh /usr/local/CPhomebrew/Cellar/cellprofiler-dev/1 (cellprofiler-dev.rb:53)

I am a novice Homebrew user, so any suggestions would be much appreciated. I have found one related closed issue, but the problem there seemed to be related to gfortran. - Thanks, AJP

Currently, when loading an image, intensities are rescaled based on the following criteria:

1. A value provided by the user*.
2. The maximum intensity supported by the image, as defined in the image metadata.
3. The maximum value supported by the image data type**.

Instead of defaulting to rescaling by supported maximum intensity (options 2 or 3), CellProfiler should default to rescaling intensities by actual maximum intensity. That is, rescale intensities between 0 and 1 such that the actual minimum intensity value is mapped to 0 and the actual maximum intensity value is mapped to 1 (and all other intensities are adjusted accordingly):

image = (image - image.min())/(image.max() - image.min())

Rescaling intensities in this way has the following benefits:

1. It works.
2. It's consistent and not dependent on image metadata.
3. It won't rescale all values to be teeny-tiny if metadata is missing and the data type is 32-bit integer (in this case, all values would be rescaled by dividing by 65535. Ew!).
4. We won't be dependent on python-bioformats to load images and rescale values (this is very important for 3D work).

* Kind of but not really. It's first scaled by supported maximum intensity (option 2 or 3) and then scaled again by the user specified value. This boolean rescale is not passed into the call to read. Nonzero values evaluate to True, so the image is first rescaled to its supported maximum intensity and then rescaled again according to the provided value.

** This isn't entirely accurate either, because these data type maximums are not correct.

Description of the "montage" module: The CP montage module could be based on our Matlab code cif_reader.m (file attached) which reads a .cif file and generates and saves the n-by-n image montages. No need to save the image montages though (displaying would be nice), as we have the CP module save images. Test files called testfile_.cif (and Matlab files from bioformats to access the cif file) are here: https://www.dropbox.com/s/ubtm77b4gnpv3bf/cif_reader_implementation.zip?dl=0 Parameters: size of the n-by-n grid, default: 32x32 grid yielding 32_32=1024 images per image montage (if you estimate that we reach the sweet spot in terms of computational speed with a larger/smaller grid then pls pick that value as default)

Background:

• A compensated image file (.cif file) is a proprietary file format of the imaging flow cytometers developed by Millipore. Typically, a .cif file contains hundreds of thousands of small images, each image has only ~7KB, and metadata such as channel number along with each image. Loading/reading a .cif file already works nicely in CP, Lee Kamentsky has written an open-source reader for cif files available on BioFormats

Why we would love to have a montage module in CP:

1. Open-source/user-friendly: We currently use Matlab for the image tiling and then load the image montages into CP, but we would like to provide an open-source and user-friendly imaging flow cytometry workflow also suitable for everyday biologists
2. speed: Running 100,000 small images in CP (segmentation, extracting features) would e.g. take about 3-4 days on my laptop (4 cores), while running the same number of images on the same machine but with image montages (currently 225 images per montage) takes 2 hours. Thus the overhead in CP is huge- with 225 images per tile we are about 30-50 times faster using the tiling as compared to handling single images. The tiling itself is fast (~seconds for typical .cif file sizes).

cif_reader.m.zip

We have now completed an initial working prototype of the RESTful image server we discussed back in May.

https://github.com/imagej/imagej-server

Please check it out and let us know what you think. We are greatly looking forward to pursuing this further so that CP and ImageJ continue to integrate well together.

While installing the developer version of CellProfiler on Mac OSX 10.6.8, I used the CPHomebrew method to install CP's dependencies. However, when I tried to run CP, I got errors about not being able to find packages. I've worked through several errors by manually installing modules (numpy, scipy, matplotlib, and a couple others), currently stuck on h5py. Unfortunately, I didn't think to report this when I first ran into the problem, and I've installed a bunch of stuff since then, so I can't reproduce the exact set of errors on this computer.

This is a pull request for the initial check in of the TrAM module.

You can add this module after tracking with TrackObjects. You will be able to select the objects to perform the computation on and their corresponding measurements to locate jumps in. There is documentation in the help window.

Let me know if you have any questions or difficulties testing. I'm happy to provide a test project and images.

Hi, I apologize if this is not the place, but I tried to install the Homebrew cellprofiler development version for mac and it failed pretty far into the process. My mac is brand new and running 10.7.3 Thanks for taking a look! -John

Here's the message from the pip.log:

/Users/jmclaughlin/CPhomebrew/bin/gfortran -Wall -arch i686 -arch x86_64 -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/scipy/fftpack/_fftpackmodule.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/drfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zrfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zfftnd.o build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/scipy/fftpack/src/dct.o build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/fortranobject.o -Lbuild/temp.macosx-10.6-intel-2.7 -ldfftpack -lfftpack -lgfortran -o build/lib.macosx-10.6-intel-2.7/scipy/fftpack/_fftpack.so

ld: library not found for -lgcc

ld: library not found for -lgcc

lipo: can't figure out the architecture type of: /var/folders/h9/mzp4k_r93pq1fzf6b9_4ljh816lyrk/T//ccPQ6k9T.out

ld: library not found for -lgcc

ld: library not found for -lgcc

lipo: can't figure out the architecture type of: /var/folders/h9/mzp4k_r93pq1fzf6b9_4ljh816lyrk/T//ccPQ6k9T.out

error: Command "/Users/jmclaughlin/CPhomebrew/bin/gfortran -Wall -arch i686 -arch x86_64 -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/scipy/fftpack/_fftpackmodule.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/drfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zrfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zfftnd.o build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/scipy/fftpack/src/dct.o build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/fortranobject.o -Lbuild/temp.macosx-10.6-intel-2.7 -ldfftpack -lfftpack -lgfortran -o build/lib.macosx-10.6-intel-2.7/scipy/fftpack/_fftpack.so" failed with exit status 1

Command /Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/bin/python -c "import setuptools;file='/Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/build/scipy/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --single-version-externally-managed --record /var/folders/h9/mzp4k_r93pq1fzf6b9_4ljh816lyrk/T/pip-rNINyf-record/install-record.txt --install-headers /Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/bin/../include/site/python2.7 failed with error code 1 in /Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/build/scipy

Exception information: Traceback (most recent call last): File "/Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/basecommand.py", line 104, in main status = self.run(options, args) File "/Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/commands/install.py", line 250, in run requirement_set.install(install_options, global_options) File "/Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/req.py", line 1133, in install requirement.install(install_options, global_options) File "/Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/req.py", line 577, in install cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False) File "/Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/init.py", line 256, in call_subprocess % (command_desc, proc.returncode, cwd)) InstallationError: Command /Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/bin/python -c "import setuptools;file='/Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/build/scipy/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --single-version-externally-managed --record /var/folders/h9/mzp4k_r93pq1fzf6b9_4ljh816lyrk/T/pip-rNINyf-record/install-record.txt --install-headers /Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/bin/../include/site/python2.7 failed with error code 1 in /Users/jmclaughlin/CPhomebrew/Cellar/cellprofiler-dev-python/1/cpdev/build/scipy

From this forum post:

"I am glad that CellProfiler installation on Linux (Ubuntu 16.04) now works (Since version 2.2.0).

The CellProfiler behaves similar to the windows in most respects, however the modules windows lacks the controls associated with zooming, panning. These controls facilitate accurate pipeline parameters like thresh-holding, bounds, smoothing scales, etc.

Is that a known issue in Linux installation, or is there a problem in my installation/JRE/JVM?"

At the end of the module's introductory help text, RelateObjects has this line: This module supports 2D and 3D objects.

Have we added this to all other modules that need it? I've not seen any other modules so far.

I've been trying to run my pipeline (with ~1000 images), but there is always a point I get an error. The individual images seem fine, but when I try to analyze them all, it can't finish (it finishes when there are like 4 images but not everything). This is from the CP e5117de. Any ideas?

Traceback (most recent call last): File "wx/_core.pyc", line 16766, in File "/Volumes/CellProfiler/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/gui/pipelinecontroller.py", line 2697, in on_stop_analysis File "/Volumes/CellProfiler/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/measurements.py", line 364, in getitem File "/Volumes/CellProfiler/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/measurements.py", line 960, in get_measurement File "/Volumes/CellProfiler/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/utilities/hdf5_dict.py", line 367, in getitem File "/Volumes/CellProfiler/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/utilities/hdf5_dict.py", line 644, in get_dataset File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_objects.c:2458) File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_objects.c:2415) File "/Volumes/CellProfiler/CellProfiler.app/Contents/Resources/lib/python2.7/h5py/_hl/group.py", line 164, in getitem File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_objects.c:2458) File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_objects.c:2415) File "h5py/h5o.pyx", line 190, in h5py.h5o.open (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/h5o.c:3507) KeyError: "Unable to open object (Can't insert duplicate key)"

From this forum post:

When the input images are named so: A - 7(fld 1 wv UV - DAPI).tif, the Metadata module is able to parse all the relevant bits (row, col, field, filters). But the Metadata_Well column in my output ends up looking like this:

A1
A10
A11
A12
A2
A3
A4
A5
A6

Which makes it very annoying to have to resort afterwards outside of CellProfiler. A suggested fix would be to add a checkbox (or toggle) in the Metadata module to normalize Well ID padding.

Possible Solution 1: checkbox that when activated makes sure all the wells have the same number of digits as the largest value (eg, A3 and A12 become A03 and A12). This change may also need to be applied to the Column metadata property (but then it would change from integer to string). Possible Solution 2: pulldown menu or radio buttons that lets you select whether to add or remove leading zeros in the numerical part of the Well metadata. If "add leading zeros" is selected, then Well (and Column?) metadata are padded to have leading zeros as needed to normalize the length. If "remove leading zeros" is selected, then any leading zeros would be removed (A03 -> A3).

Is your feature request related to a problem? Please describe. I've been informed that running a pipeline containing SaveCroppedObjects on S3 fails since it generates invalid paths, and so "Prefix saved crop image name with input image name" cannot be used.

Describe the solution you'd like I suggest modifying the settings to mirror those in SaveImages, particularly the following:

• Select method for constructing file names
• Select image name for file prefix
• Enter single file name
• Append a suffix to the image file name?
• Text to append to the image name

Describe alternatives you've considered Since we use an analysis platform which uses LoadData-enabled pipelines, we considered having the backend rename the images according to the Barcode/Row/Column/Field columns in the image list CSV. However, a change to the settings themselves would be easier to handle.

Additional context The S3 issue may be a bug so I'm happy to open an issue against that specifically. However, to be clear, this feature request concerns a change to settings to mitigate the problem.

I followed the exact same steps as mentioned in the on git page of installation Desktop (please complete the following information):

• OS: Ubuntu 20.04
• CellProfiler Version 4.0.0rc12
• Downloaded from the website or installed from source? from source
• If the latter, how (cloned, installed from pypi, etc)? cloned it as mentioned in link above

Describe the issue A clear and concise description of which steps of the installation succeeded and which failed. Installation is failing with the following error

After running the pipeline there is an error about exporttospreadsheet and there is only one csv file showing in the chosen output area, but usually there should be many

Has anyone encountered that before, any suggestions are appreciated!

Resolves #4653

ExportToSpreadsheet gets mad trying to say "what is the biggest number objects that I have in this image" if there are object creation modules that are skipped.