OpenCV-Erlang/Elixir bindings

Related tags

Computer Visionopencvelixirelixir-libraryerlang-library
Overview

evision [WIP]

: OS : arch Build Status
Ubuntu 20.04 arm64 CI
Ubuntu 20.04 armv7 CI
Ubuntu 20.04 s390x CI
Ubuntu 20.04 ppc64le CI
Ubuntu 20.04 x86_64 CI
macOS 11 Big Sur x86_64 CI

Apple Silicon M1-series are supported, actually I coded and tested this project on my M1 Max MacBook Pro, but M1 GitHub Action runners are not yet available.

Nerves Support

Nerves

Prebuilt firmwares are available here. Select the most recent run and scroll down to the Artifacts section, download the firmware file for your board and run

fwup /path/to/the/downloaded/firmware.fw

SSH keys can be found in nerves/id_rsa[.pub]. For obvious security reason, please use these prebuilt firmwares for evaluation only.

Description

evision will pull OpenCV source code from GitHub, then parse and automatically generate corresponding OpenCV-Elixir bindings.

This project uses and modifies gen2.py and hdr_parser.py from the python module in the OpenCV repo so that they output header files that can be used in Elixir bindings.

We hope this project can largely reduce the work of manually porting OpenCV functions/modules to Elixir.

Current available modules:

  • calib3d
  • core
  • features2d
  • flann
  • highgui
  • imgcodecs
  • imgproc
  • ml
  • photo
  • stitching
  • ts
  • video
  • videoio

Note, edit config/config.exs to enable/disable OpenCV modules and image coders.

Dependencies

Required

  • Python3 (Only during the compilation, to generate binding files)
  • CMake
  • Erlang development library/headers.

Optional

  • curl/wget. To download OpenCV source zip file.

    Optional if you put the source zip file to 3rd_party/cache/opencv-${OPENCV_VER}.zip.

  • unzip. To unzip the OpenCV source zip file.

    Optional if you supply OpenCV source code at 3rd_party/opencv/opencv-${OPENCV_VER}.

Installation

In order to use evision, you will need Elixir installed. Then create an Elixir project via the mix build tool:

$ mix new my_app

Then you can add evision as dependency in your mix.exs. At the moment you will have to use a Git dependency while we work on our first release:

def deps do
  [
    {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
  ]
end

Note

  • Use MAKE_BUILD_FLAGS="-j$(nproc)" environment variable to set number of jobs for compiling.

    Default value: "-j#{System.schedulers_online()}". In mix.exs.

  • Use TOOLCHAIN_FILE="/path/to/toolchain.cmake" to set your own toolchain.

    Default value: "nerves/toolchain.cmake". In Makefile.

  • Edit config/config.exs to enable/disable OpenCV modules and image coders.

  • Some useful commands

    MIX_ENV=dev
OPENCV_VER=4.5.4
MIX_TARGET=rpi4

# delete OpenCV related CMake build caches.
rm -rf "_build/${MIX_ENV}/lib/evision/cmake_opencv_${OPENCV_VER}"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/cmake_opencv_${OPENCV_VER}"

# remove downloaded OpenCV source zip file.
rm -f "3rd_party/cache/opencv-${OPENCV_VER}"

# delete evision.so (so that `make` can rebuild it, useful when you manually modified C/C++ source code)
rm -f "_build/${MIX_ENV}/lib/evision/priv/evision.so"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/priv/evision.so"

# delete evision related CMake build caches.
rm -rf "_build/${MIX_ENV}/lib/evision/cmake_evision"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/cmake_evision"

Current Status

{:ok, gray_mat} = OpenCV.imread("/path/to/img.png", flags: OpenCV.cv_imread_grayscale)
{:ok, gray_blur_mat} = OpenCV.blur(gray_mat, [10,10], anchor: [1,1])
{:ok, colour_mat} = OpenCV.imread("/path/to/img.png")
{:ok, colour_blur_mat} = OpenCV.blur(colour_mat, [10,10], anchor: [1,1])
:ok = OpenCV.imwrite("/path/to/img-gray-and-blur.png", gray_blur_mat)
:ok = OpenCV.imwrite("/path/to/img-colour-and-blur.png", colour_blur_mat)

{:ok, cap} = OpenCV.VideoCapture.videocapture(0)
{:ok, cap_mat} = OpenCV.VideoCapture.read(cap)
:ok = OpenCV.imwrite("/path/exists/capture-mat.png", cap_mat)
:error = OpenCV.imwrite("/path/not/exists/capture-mat.png", cap_mat)

Todo

  • Update .py files in py_src so that they output header files for Erlang bindings.

  • Automatically generate erl_cv_nif.ex.

  • Automatically generate opencv_*.ex files using Python.

  • Automatically convert enum constants in C++ to "constants" in Elixir

  • When a C++ function's return value's type is bool, map true to :ok and false to :error.

    # not this
{:ok, true} = OpenCV.imwrite("/path/to/save.png", mat, [])
# but this
:ok = OpenCV.imwrite("/path/to/save.png", mat, [])

  • Make optional parameters truly optional.

    # not this
{:ok, cap} = OpenCV.VideoCapture.videocapture(0, [])
# but this
{:ok, cap} = OpenCV.VideoCapture.videocapture(0)
# this also changes the above example to
:ok = OpenCV.imwrite("/path/to/save.png", mat)

  • Nerves support (rpi4 for now).

  • Add OpenCV.Mat module.

    {:ok, type} = OpenCV.Mat.type(mat)
{:ok, {:u, 8}}

{:ok, {height, weight, channel}} = OpenCV.Mat.shape(mat)
{:ok, {1080, 1920, 3}}
{:ok, {height, weight}} = OpenCV.Mat.shape(gray_mat)
{:ok, {1080, 1920}}

{:ok, bin_data} = OpenCV.Mat.to_binary(mat)
{:ok, << ... binary data ... >>}

  • Add support for Nx.

    nx_tensor = OpenCV.Mat.to_nx(mat)
#Nx.Tensor<
   u8[1080][1920][3]
   [[ ... pixel data ... ]]
>
{:error, reason} = OpenCV.Mat.to_nx(invalid_mat)

  • Edit config/config.exs to enable/disable OpenCV modules and image coders.

  • Add tests.

How does this work?

  1. This project will first pull OpenCV source code from git (as git submodules).

  2. Inside the OpenCV project, there is an opencv-python module, 3rd_party/opencv/modules/python. If the opencv-python module is enabled,

    cmake ...
    -D PYTHON3_EXECUTABLE=$(PYTHON3_EXECUTABLE) \
    ...

    It will generate files for opencv-python bindings in cmake build dir, cmake_build_dir/modules/python_bindings_generator.

    We are interested in the headers.txt file as it tells us which headers should we parse (this header list changes based on the enabled modules).

    We also need to check another file, pyopencv_custom_headers.h. This file includes pyopencv compatible headers from modules that need special handlings to enable interactions with Python. We will talk about this later.

  3. Originally, the headers.txt file will be passed to 3rd_party/opencv/modules/python/src2/gen2.py and that Python script will then generate pyopencv_*.h in cmake_build_dir/modules/python_bindings_generator. Here we copy that Python script and modify it so that it outputs c_src/evision_*.h files which use c_src/erlcompat.hpp and c_src/nif_utils.hpp to make everything compatible with Erlang.

  4. c_src/opencv.cpp includes almost all specialised and generic evision_to and evision_from functions. They are used for making conversions between Erlang and C++. Some conversion functions are defined in module custom headers.

  5. This is where we need to make some changes to pyopencv_custom_headers.h. We first copy it to c_src/evision_custom_headers.h and copy every file it includes to c_src/evision_custom_headers/. Then we make corresponding changes to c_src/evision_custom_headers/*.hpp files so that these types can be converted from and to Erlang terms. The header include path in c_src/evision_custom_headers.h should be changed correspondingly.

  6. However, it is hard to do step 5 automatically. We can try to create a PR which puts these changed files to the original OpenCV repo's {module_name}/mics/erlang/ directory. Now we just manually save them in c_src/evision_custom_headers. Note that step 5 and 6 are done manually, calling py_src/gen2.py will not have effect on c_src/evision_custom_headers.h and *.hpp files in c_src/evision_custom_headers.

  7. Another catch is that, while function overloading is easy in C++ and optional arguments is simple in Python, they are not quite friendly to Erlang/Elixir. There is basically no function overloading in Erlang/Elixir. Although Erlang/Elixir support optional argument (default argument), it also affects the function's arity and that can be very tricky to deal with. For example,

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts \\ []), do: :nil
  def open(self, filename), do: :nil
  # ... other functions ...
end

    In this case, def open(self, camera_index, opts \\ []), do: :nil will define open/3 and open/2 at the same time. This will cause conflicts with def open(self, filename), do: :nil which defines open/2.

    So we cannot use default arguments. Now, say we have

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts), do: :nil
  def open(self, filename, opt), do: :nil
end

    Function overloading in C++ is relatively simple as compiler does that for us. In this project, we have to do this ourselves in the Erlang/Elixir way. For the example above, we can use guards.

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts) when is_integer(camera_index) do
    # do something
  end
  def open(self, filename, opt) when is_binary(filename) do
    # do something
  end
end

    But there are some cases we cannot distinguish the argument type in Erlang/Elixir becauase they are resources (instance of a certain C++ class).

    defmodule OpenCV.SomeModule do
  # @param mat: Mat
  def func_name(mat) do
    # do something
  end

  # @param mat: UMat
  def func_name(mat) do
    # do something
  end
end

    In such cases, we only keep one definition. The overloading will be done in c_src/opencv.cpp (by evision_to).

  8. Enum handling. Originally, PythonWrapperGenerator.add_const in py_src/gen2.py will be used to handle those enum constants. They will be saved to a map with the enum's string representation as the key, and, of course, enum's value as the value. In Python, when a user uses the enum, say cv2.COLOR_RGB2BGR, it will perform a dynamic lookup which ends up calling corresponding evision_[to|from]. evision_[to|from] will take the responsibility to convert between the enum's string representation and its value. Although in Erlang/Elixir we do have the ability to both create atoms and do the similar lookups dynamically, the problem is that, if an enum is used as one of the arguments in a C++ function, it may be written as void func(int enum) instead of void func(ENUM_TYPE_NAME enum). However, to distinguish between overloaded functions, some types (int, bool, string, char, vector) will be used for guards. For example, void func(int enum) will be translated to def func(enum) when is_integer(enum), do: :nil. Adding these guardians help us to make some differences amongst overloaded functions in step 7. However, that prevents us froming passing an atom to def func(enum) when is_integer(enum), do: :nil. Technically, we can add one more variant def func(enum) when is_atom(enum), do: :nil for this specific example, but there are tons of functions has one or more ints as their input arguments, which means the number of variants in Erlang will increase expoentially (for each int in a C++ function, it can be either a real int or an enum). Another way is just allow it to be either an integer or an atom:

    def func(enum) when is_integer(enum) or is_atom(enum) do
  :nil
end

    But in this way, atoms are created on-the-fly, users cannot get code completion feature for enums from their IDE. But, finally, we have a good news that, in Erlang/Elixir, when a function has zero arguments, you can write its name without explictly calling it, i.e.,

    defmodule M do
  def enum_name(), do: 1
end

1 = M.enum_name
1 = M.enum_name()

    So, in this project, every enum is actually transformed to a function that has zero input arguments.

How do I make it compatible with more OpenCV modules?

Because of the reason in step 6, when you enable more modules, if that module has specialised custom header for python bindings, the custom headers will be added to cmake_build_dir/modules/python_bindings_generator/pyopencv_custom_headers.h. Then you can manually copy corresponding specialised custom headers to c_src/evision_custom_headers and modify these conversion functions in them.

Acknowledgements

  • gen2.py, hdr_parser.py and c_src/erlcompat.hpp were directly copied from the python module in the OpenCV repo. Changes applied.
  • Makefile, CMakeLists.txt and c_src/nif_utils.hpp were also copied from the torchx module in the elixir-nx repo. Minor changes applied.
Comments
  • Proposal: bridging with Nx

    Proposal: bridging with Nx

    I'm developing Excv, a bridge between Nx and OpenCV: https://github.com/zeam-vm/excv

    I hope that evision will be integrated with such bridging. I'd like to contribute to evision if you wish so. Would you please inform me where the #Reference structure includingMat?

    opened by zacky1972 14
  • Share image data between Vix and eVision (and Nx)

    Share image data between Vix and eVision (and Nx)

    I am the author of the Image library that is only possible because of @akash-akya's Vix which is a binding to libvips. Given Vix and eVision have similar problem domains, and very similar matrix implementations, I would like to be able to share image data between the two libraries. This would allow Image (and of course any other library) to leverage the capabilities of both Vix and eVision (and very desirably Nx) in a very memory efficient and performant manner.

    Objectives

    1. Share image data without copying the matrix (I think a shared interpretation is possible) - but need to make sure we know who has memory ownership and who is managing the memory (and I'm not good at this part).
    2. Having a way to agree and share data formats (ie a common lexicon to refer to image data type and matrix shape - perhaps using the Nx terminology?)
    3. Act as a model of how to do the same integration with Nx. I know eVision has Nx integration but it appears to be based upon converting to and from Elixir binaries - which I'd prefer not to do since lots of NIF copying would contradict the memory efficiency and performance objectives.

    I'm looking for guidance, suggestions and ideas - and hopefully someone motivated to tackle this because its a bit beyond my competence. :-)

    Given the intent to also provide integration with Nx, any comments from @josevalim would also be very welcome.

    enhancement 
    opened by kipcole9 13
  • Import Nx tensor that is in {width, height, channels} shape with RGB data format

    Import Nx tensor that is in {width, height, channels} shape with RGB data format

    Cocoa, I have tried everything I can think of to take an Nx tensor that is of the shape {width, height, channels} that contains RGB format data, and convert it in eVision to {height, width, channels} in BGR format.

    Any chance I might ask for your advice and recommendations?

    What I've tried

    Here is what I have tried which is, I think, the required process but the saved image is definitely not what is expected!

    tensor = File.read!("path_to/color_checker.etf") |> :erlang.binary_to_term()
{:ok, mat} = Evision.Nx.to_mat(tensor)
{:ok, transposed} = Evision.Mat.transpose(mat, [1, 0, 2])
{:ok, bgr} = Evision.cvtColor(transposed, Evision.cv_COLOR_RGB2BGR())
Evision.imwrite "some_path/color_checker.jpg", bgr

    I have followed the converse process in Image and the results line up with expectations. Which is not too surprising since Image expects data in {width, height, channels} and RGB format. I mention this just to note that I have verified that the tensor does represent the underlying example image.

    Artifacts

    The image converted to a tensor and stored with :erlang.term_to_binary/1 and zipped: color_checker.etf.zip

    The original image: color_checker

    opened by kipcole9 10
  • cropping of image using ROI not working

    cropping of image using ROI not working

    Please refer to the below images to understand about the issue. This is a function used to get image: Screenshot from 2022-10-10 10-50-12 This is the error window: Screenshot from 2022-10-10 10-52-21 Its working in python for reference: Screenshot from 2022-10-10 10-52-55

    opened by navrkr 9
  • Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

    Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

    Hi,

    I did the following steps:

    mix new evision_test
cd evision_test

    Then, I added evision to mix.exs as follows:

      defp deps do
    [
      {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
    ]
  end

    Then, I did the following steps:

    mix deps.get
mix compile

    Then, I got the following error:

    ==> evision
[/Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip or
        /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.zip, and cannot find /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.ZIP, period.
make: *** [/Users/zacky/evision_test/deps/evision/3rd_party/opencv/opencv-4.5.5/modules/core/include/opencv2/core/utils/configuration.private.hpp] Error 9
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    Then, I did the following steps:

    mix deps.compile evision

    Then, I got the following fatal error:

    ==> evision
-- The CXX compiler identification is AppleClang 13.0.0.13000029
-- The C compiler identification is AppleClang 13.0.0.13000029
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
(snip)
-- General configuration for OpenCV 4.5.5 =====================================
--   Version control:               ca737c9
-- 
--   Platform:
--     Timestamp:                   2022-01-26T09:38:53Z
--     Host:                        Darwin 21.2.0 arm64
--     CMake:                       3.22.1
--     CMake generator:             Unix Makefiles
--     CMake build tool:            /usr/bin/make
--     Configuration:               RELEASE
-- 
--   CPU/HW features:
--     Baseline:                    NEON FP16
-- 
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /Library/Developer/CommandLineTools/usr/bin/c++  (ver 13.0.0.13000029)
--     C++ flags (Release):         -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /Library/Developer/CommandLineTools/usr/bin/cc
--     C flags (Release):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,-dead_strip  
--     Linker flags (Debug):        -Wl,-dead_strip  
--     ccache:                      NO
--     Precompiled headers:         NO
--     Extra dependencies:
--     3rdparty dependencies:
-- 
--   OpenCV modules:
--     To be built:                 calib3d core dnn features2d flann highgui imgcodecs imgproc ml objdetect photo python2 stitching ts video videoio
--     Disabled:                    gapi world
--     Disabled by dependency:      -
--     Unavailable:                 java python3
--     Applications:                perf_tests apps
--     Documentation:               NO
--     Non-free algorithms:         NO
-- 
--   GUI:                           COCOA
--     Cocoa:                       YES
--     VTK support:                 NO
-- 
--   Media I/O: 
--     ZLib:                        zlib (ver 1.2.11)
--     JPEG:                        build-libjpeg-turbo (ver 2.1.2-62)
--     WEBP:                        build (ver encoder: 0x020f)
--     PNG:                         libpng (ver 1.6.37)
--     TIFF:                        libtiff (ver 42 / 4.2.0)
--     JPEG 2000:                   build (ver 2.4.0)
--     OpenEXR:                     OpenEXR::OpenEXR (ver 3.1.3)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
-- 
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                YES (4.0.0)
--     GStreamer:                   NO
--     AVFoundation:                YES
-- 
--   Parallel framework:            GCD
-- 
--   Trace:                         YES (with Intel ITT)
-- 
--   Other third-party libraries:
--     Lapack:                      YES (/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/Accelerate.framework -lm -ldl)
--     Eigen:                       YES (ver 3.4.0)
--     Custom HAL:                  YES (carotene (ver 0.0.1))
--     Protobuf:                    build (3.19.1)
-- 
--   OpenCL:                        YES (no extra features)
--     Include path:                NO
--     Link libraries:              -framework OpenCL
-- 
--   Python 2:
--     Interpreter:                 /usr/bin/python2.7 (ver 2.7.18)
--     Libraries:                   /usr/lib/libpython2.7.dylib (ver 2.7.18)
--     numpy:                       /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include (ver 1.8.0rc1)
--     install path:                lib/python2.7/site-packages/cv2/python-2.7
-- 
--   Python (for build):            /usr/bin/python2.7
-- 
--   Java:                          
--     ant:                         NO
--     JNI:                         NO
--     Java wrappers:               NO
--     Java tests:                  NO
-- 
--   Install to:                    /Users/zacky/evision_test/_build/dev/lib/evision/priv
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5
[  0%] Building C object 3rdparty/ittnotify/CMakeFiles/ittnotify.dir/src/ittnotify/ittnotify_static.c.o
[  0%] Generate opencv4.pc
[  0%] Building C object 3rdparty/zlib/CMakeFiles/zlib.dir/adler32.c.o
[  0%] Building C object 3rdparty/openjpeg/openjp2/CMakeFiles/libopenjp2.dir/thread.c.o
(snip)
[ 72%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/vkcom/vulkan/vk_loader.cpp.o
[ 72%] Linking CXX shared library ../../lib/libopencv_dnn.dylib
[ 72%] Built target opencv_dnn
make[1]: *** [all] Error 2
make: *** [/Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 2
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    My environment is as follows:

    $ uname -a 
Darwin zackym1air.local 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:29:10 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T8101 arm64
$ elixir -v
Erlang/OTP 24 [erts-12.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Elixir 1.13.2 (compiled with Erlang/OTP 24)

    Thank you.

    opened by zacky1972 9
  • Evision.DNN.DetectionModel can't setInputParams at YOLOv4

    Evision.DNN.DetectionModel can't setInputParams at YOLOv4

    I tried to use YOLOv4 at Evision.DNN.DetectionModel

    dog = "yolov4/dog.jpg"
weights = "yolov4/yolov4.weights"
config = "yolov4/yolov4.cfg"
classes = "yolov4/coco.names"
mat = Evision.imread(dog)

Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.DetectionModel.detect(mat)

    return error

    {:error,
 "OpenCV(4.6.0) /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/dnn/src/model.cpp:98: error: (-201:Incorrect size of input array) Input size not specified in function 'processFrame'\n"}

    i guess need such as python code

    model.setInputParams(1.0, (416, 416), (0.0, 0.0, 0.0), true, false)

    but

    Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams(scale: 1.0, size: {416, 416}, mean: {0, 0, 0}, swapRB: true, crop: false)

    ** (UndefinedFunctionError) function Evision.DNN.Model.setInputParams/2 is undefined or private. Did you mean:

    Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams({416, 416})

    {:error, "cannot get `cv::dnn::Model` from `self`: mismatched type or invalid resource?"}

    How about the following solution? 1 Convert the loaded model to Evision.DNN.Model 2 Enable setInputParams in DetectionModel as well

    Evision.DNN.readFromDarknet load and detect are success but, I wan't to impl post process :-(

    Enviroments evision 0.1.16 elixir 1.14.1 elrang 25.0.1 os macOS 12.6

    opened by thehaigo 8
  • small doubt regarding contours in evision

    small doubt regarding contours in evision

    i get the values of findContours after getting is their any method to seperate contours using in built function like in python imutil.grabcontours

    the code is here

    `def grab_contours(cnts):
    # if the length the contours tuple returned by cv2.findContours
    # is 2 then we are using either OpenCV v2.4, v4-beta, or
    # v4-official
    if len(cnts) == 2:
        cnts = cnts[0]

    # if the length of the contours tuple is 3 then we are using
    # either OpenCV v3, v4-pre, or v4-alpha
    elif len(cnts) == 3:
        cnts = cnts[1]

    # otherwise OpenCV has changed their cv2.findContours return
    # signature yet again and I have no idea WTH is going on
    else:
        raise Exception(("Contours tuple must have length 2 or 3, "
            "otherwise OpenCV changed their cv2.findContours return "
            "signature yet again. Refer to OpenCV's documentation "
            "in that case"))

    # return the actual contours array
    return cnts`

    could please help us sir @cocoa-xu @josevalim

    opened by Jagan3534 8
  • An error occur when compiling on Jetson (Linux, aarch64)

    An error occur when compiling on Jetson (Linux, aarch64)

    Hi,

    I compiled evision on NVIDIA Jetson AGX Xavier (Ubuntu 18.04 Linux Kernel 4.9.201-tegra, aarch64), but the following error occurred:

    ==> evision
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   117    0   117    0     0    393      0 --:--:-- --:--:-- --:--:--   393
100 89.8M    0 89.8M    0     0  9401k      0 --:--:--  0:00:09 --:--:-- 14.4M
CMake Error: The source directory "/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/BUILD_OPENEXR=ON" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
Makefile:96: recipe for target '/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt' failed
make: *** [/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 1
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. If you are using
Ubuntu or any other Debian-based system, install the packages
"build-essential". Also install "erlang-dev" package if not
included in your Erlang/OTP version. If you're on Fedora, run
"dnf group install 'Development Tools'".

    Are Linux and aarch64 not supported?

    opened by zacky1972 8
  • imshow function is broken in pre-compiled library.

    imshow function is broken in pre-compiled library.

    I recently updated from elixir 1.0.0-dev to 1.0.9 version of Evision and everything is great till now, just tried getting the image as output and this error popped (Note: All the issue is with cvwait and imshow doesn't work without cvwait).

    IMG_20221011_070312_455

    opened by navrkr 7
  • sorry for what is did . we have a small problem in sort sir

    sorry for what is did . we have a small problem in sort sir

    I apologise if i sound rude, i did not know english that well may be this is why i sounds rude.

    here same we got the error in the elixir in python we wrote like this in elixir we tried to give but in sort 2 nd parameter is doubted sir in Evision hexdoxs we tried to see the example but in new version they dont show contours = sorted(contours, key=cv2.contourArea, reverse=True)

    could you please help us

    opened by Jagan3534 6
  • Can't round trip QRcode encode/decode

    Can't round trip QRcode encode/decode

    QRcode detection is working well (as far as I can test) but when I generate a QRcode and attempt to detect it I am not having any luck. For example:

    iex> qrcode = Evision.QRCodeEncoder.encode Evision.QRCodeEncoder.create(), "This is a string"           
%Evision.Mat{
  channels: 1,
  dims: 2,
  type: {:u, 8},
  raw_type: 0,
  shape: {25, 25},
  ref: #Reference<0.1002129465.3225026580.34308>
}
iex> Evision.QRCodeDetector.detectAndDecode Evision.QRCodeDetector.qrCodeDetector(), qrcode             
{"",
 %Evision.Mat{
   channels: 2,
   dims: 2,
   type: {:f, 32},
   raw_type: 13,
   shape: {1, 4, 2},
   ref: #Reference<0.1002129465.3225026580.34310>
 }, {:error, "empty matrix"}}
    1. The encoded image is very small, {25, 25} and I don't actually know how - even if its possible - to define parameters for the encoder to make the image larger.
    2. I can see there is a module Evision.QRCodeEncoder.Params but I'm not having any success try to define a param structure. For example:
    iex> Evision.QRCodeEncoder.Params.get_mode Evision.QRCodeEncoder.Params.qrcodeencoder_params()
zsh: segmentation fault  iex -S mix

    Thanks for your patience @cocoa-xu while I work out have to navigate the wonderful world of OpenCV/eVision - help and guidance much appreciated.

    opened by kipcole9 6
  • [Feature] opencv-contrib modules

    [Feature] opencv-contrib modules

    I plan to add support for modules in opencv_contrib gradually. Of course, this would bring an inevitable question: not everyone will use modules in opencv_contrib, so including them in the precompiled binaries will increase the footprint.

    Will it be feasible to have an :evision_contrib library in Elixir (like its python counterpart, where we have opencv-python and opencv-python-contrib)?

    Modules (part 1)

    • [x] barcode
    • [x] dnn_superres
    • [x] face
    • [x] img_hash
    • [x] plot
    • [x] quality
    • [x] rapid
    • [x] sfm
    • [x] shape
    • [x] structured_light
    • [x] text
    • [x] tracking
    • [x] xfeatures2d
    • [x] ximgproc
    • [x] xphoto

    Modules (part 2)

    • [x] aruco
    • [x] bgsegm
    • [x] bioinspired
    • [x] hfs
    • [x] line_descriptor
    • [x] mcc
    • [x] reg
    • [x] rgbd
    • [x] saliency
    • [x] stereo
    • [x] surface_matching
    • [x] wechat_qrcode

    Modules (part 3)

    • [x] cudaarithm
    • [x] cudabgsegm
    • [x] cudacodec
    • [x] cudafeatures2d
    • [x] cudafilters
    • [x] cudaimgproc
    • [x] cudalegacy
    • [x] cudaobjdetect
    • [x] cudaoptflow
    • [x] cudastereo
    • [x] cudawarping
    • [x] cudev

    todo

    • [x] Check module name and struct name
    • [ ] Check _Params classes
    • [ ] Examples
    • [ ] Tests
    enhancement 
    opened by cocoa-xu 2
  • [Feature-GUI] Implementing `Evision.Wx`

    [Feature-GUI] Implementing `Evision.Wx`

    As mentioned in #103:

    Even compiling OpenCV with highgui module, sometimes the Evision.HighGui.imshow function does not really work as expected (in evision): the window(s) it opened will not respond to any GUI events until calling Evision.HighGui.waitKey again.

    To make it as good as one would expect it to be, like how it is in C++/Python, perhaps it would require one to invest a few weeks of time for it. But I'm focusing on other aspects of this library right now, also I'm hesitating to do it because I don't know if this would worth that amount time.

    Also, as :wx has official support from Erlang, and it's well tested, and one can even build a relatively complex GUI app with :wx if they want.

    This module will try to make itself easy to use. Meanwhile, the Evision.HighGui module will stay in the source code even after Evision.Wx is completed.

    Here is a non-exhaustive list of desired/prospective functions in Evision.Wx. All of them, including the type and the number of their input arguments, are subject to changes in the future, or will not be implemented if the desired functionality is partially or wholly impossible to achieve in Erlang/Elixir with :wx.

    Functions available in OpenCV's highgui module can be found here.

    Any contributions or suggestions (either about Evision.Wx or Evision.HighGui) are more than welcomed - this also applies to the whole project. :)

    enhancement 
    opened by cocoa-xu 0
  • Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

    Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

    cv::Mat (and its variants) is the OpenCV's implementation of multi-dimensional array, so technically, it can be used as a backend for the Nx (numerical-elixir).

    opened by cocoa-xu 8
  • OpenCV gapi module

    OpenCV gapi module

    Based on the contents of pyopencv_gapi.hpp, it seems that the binding code will need to call an Elixir function and fetch the result.

    This perhaps could be done if we send related ERL_NIF_TERMs to a separate Erlang thread and send the result back. But it's not easy to properly design and write the whole thing.

    Maybe it's not worth the effort... I probably won't port this module anytime soon.

    This module is highly likely to be excluded from evision 0.1.0.

    opened by cocoa-xu 0
Releases(v0.1.25)
Owner
Cocoa
Ph.D student @ UofG
Cocoa
GitHub Repository
A program that takes in the hand gesture displayed by the user and translates ASL.

Interactive-ASL-Recognition Using the framework mediapipe made by google, OpenCV library and through self teaching, I was able to create a program tha

Riddhi Bajaj 3 Nov 22, 2021
Awesome anomaly detection in medical images

A curated list of awesome anomaly detection works in medical imaging, inspired by the other awesome-* initiatives.

Kang Zhou 57 Dec 19, 2022
This is used to convert a string to an Image with Handwritten Characters.

Text-to-Handwriting-using-python This is used to convert a string to an Image with Handwritten Characters. text_to_handwriting(string: str, save_to: s

Akashdeep Mahata 3 Aug 15, 2022
list all open dataset about ocr.

ocr-open-dataset list all open dataset about ocr. printed dataset year Born-Digital Images (Web and Email) 2011-2015 COCO-Text 2017 Text Extraction fr

hongbomin 95 Nov 24, 2022
Face Anonymizer - FaceAnonApp v1.0

Face Anonymizer - FaceAnonApp v1.0 Blur faces from image and video files in /data/files folder. Contents Repo of the source files for the FaceAnonApp.

6 Apr 18, 2022
Slice a single image into multiple pieces and create a dataset from them

OpenCV Image to Dataset Converter Slice a single image of Persian digits into mu

Meysam Parvizi 14 Dec 29, 2022
Deskewing images with slanted content

skew_correction De-skewing images with slanted content by finding the deviation using Canny Edge Detection. To Run: In python 3.6, from deskew import

13 Aug 27, 2022
A selectional auto-encoder approach for document image binarization

The code of this repository was used for the following publication. If you find this code useful please cite our paper: @article{Gallego2019, title =

Javier Gallego 89 Nov 18, 2022
A Tensorflow model for text recognition (CNN + seq2seq with visual attention) available as a Python package and compatible with Google Cloud ML Engine.

Attention-based OCR Visual attention-based OCR model for image recognition with additional tools for creating TFRecords datasets and exporting the tra

Ed Medvedev 933 Dec 29, 2022
Primary QPDF source code and documentation

QPDF QPDF is a command-line tool and C++ library that performs content-preserving transformations on PDF files. It supports linearization, encryption,

QPDF 2.2k Jan 04, 2023
Super Mario Game With Python

Super_Mario Hello all this is a simple python program which tries to use our body as a controller for the super mario game Here I have used media pipe

Adarsh Badagala 219 Nov 25, 2022
SCOUTER: Slot Attention-based Classifier for Explainable Image Recognition

SCOUTER: Slot Attention-based Classifier for Explainable Image Recognition PDF Abstract Explainable artificial intelligence has been gaining attention

87 Dec 26, 2022
This can be use to convert text in a file to handwritten text.

TextToHandwriting This can be used to convert text to handwriting. Clone this project or download the code. Run TextToImage.py give the filename of th

Ashutosh Mahapatra 2 Feb 06, 2022
OCR, Object Detection, Number Plate, Real Time

README.md PrePareded anaconda env requirements.txt clova AI → deep text recognition → trained weights (ex, .pth) wpod-net weights (ex, .h5 , .json) ht

Kaven Lee 7 Dec 06, 2022
Application that instantly translates sign-language to letters.

Sign Language Translator Project Description The main purpose of project is translating sign-language to letters. In accordance with this purpose we d

3 Sep 29, 2022
A facial recognition device is a device that takes an image or a video of a human face and compares it to another image faces in a database.

A facial recognition device is a device that takes an image or a video of a human face and compares it to another image faces in a database. The structure, shape and proportions of the faces are comp

Pavankumar Khot 4 Mar 19, 2022
WACV 2022 Paper - Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matching

Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matching Code based on our WACV 2022 Accepted Paper: https://arxiv.org/pdf/

Andres 13 Dec 17, 2022
A Python script to capture images from multiple webcams at once and save them into your local machine

Capturing multiple images at once from Webcam Using OpenCV Capture multiple image by accessing the webcam of your system and save it to your machine.

Fazal ur Rehman 2 Apr 16, 2022
Extract tables from scanned image PDFs using Optical Character Recognition.

ocr-table This project aims to extract tables from scanned image PDFs using Optical Character Recognition. Install Requirements Tesseract OCR sudo apt

Abhijeet Singh 209 Dec 06, 2022
Run tesseract with the tesserocr bindings with @OCR-D's interfaces

ocrd_tesserocr Crop, deskew, segment into regions / tables / lines / words, or recognize with tesserocr Introduction This package offers OCR-D complia

OCR-D 38 Oct 14, 2022