OpenCV-Erlang/Elixir bindings

Related tags

Computer Visionopencvelixirelixir-libraryerlang-library
Overview

evision [WIP]

: OS : arch Build Status
Ubuntu 20.04 arm64 CI
Ubuntu 20.04 armv7 CI
Ubuntu 20.04 s390x CI
Ubuntu 20.04 ppc64le CI
Ubuntu 20.04 x86_64 CI
macOS 11 Big Sur x86_64 CI

Apple Silicon M1-series are supported, actually I coded and tested this project on my M1 Max MacBook Pro, but M1 GitHub Action runners are not yet available.

Nerves Support

Nerves

Prebuilt firmwares are available here. Select the most recent run and scroll down to the Artifacts section, download the firmware file for your board and run

fwup /path/to/the/downloaded/firmware.fw

SSH keys can be found in nerves/id_rsa[.pub]. For obvious security reason, please use these prebuilt firmwares for evaluation only.

Description

evision will pull OpenCV source code from GitHub, then parse and automatically generate corresponding OpenCV-Elixir bindings.

This project uses and modifies gen2.py and hdr_parser.py from the python module in the OpenCV repo so that they output header files that can be used in Elixir bindings.

We hope this project can largely reduce the work of manually porting OpenCV functions/modules to Elixir.

Current available modules:

  • calib3d
  • core
  • features2d
  • flann
  • highgui
  • imgcodecs
  • imgproc
  • ml
  • photo
  • stitching
  • ts
  • video
  • videoio

Note, edit config/config.exs to enable/disable OpenCV modules and image coders.

Dependencies

Required

  • Python3 (Only during the compilation, to generate binding files)
  • CMake
  • Erlang development library/headers.

Optional

  • curl/wget. To download OpenCV source zip file.

    Optional if you put the source zip file to 3rd_party/cache/opencv-${OPENCV_VER}.zip.

  • unzip. To unzip the OpenCV source zip file.

    Optional if you supply OpenCV source code at 3rd_party/opencv/opencv-${OPENCV_VER}.

Installation

In order to use evision, you will need Elixir installed. Then create an Elixir project via the mix build tool:

$ mix new my_app

Then you can add evision as dependency in your mix.exs. At the moment you will have to use a Git dependency while we work on our first release:

def deps do
  [
    {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
  ]
end

Note

  • Use MAKE_BUILD_FLAGS="-j$(nproc)" environment variable to set number of jobs for compiling.

    Default value: "-j#{System.schedulers_online()}". In mix.exs.

  • Use TOOLCHAIN_FILE="/path/to/toolchain.cmake" to set your own toolchain.

    Default value: "nerves/toolchain.cmake". In Makefile.

  • Edit config/config.exs to enable/disable OpenCV modules and image coders.

  • Some useful commands

    MIX_ENV=dev
OPENCV_VER=4.5.4
MIX_TARGET=rpi4

# delete OpenCV related CMake build caches.
rm -rf "_build/${MIX_ENV}/lib/evision/cmake_opencv_${OPENCV_VER}"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/cmake_opencv_${OPENCV_VER}"

# remove downloaded OpenCV source zip file.
rm -f "3rd_party/cache/opencv-${OPENCV_VER}"

# delete evision.so (so that `make` can rebuild it, useful when you manually modified C/C++ source code)
rm -f "_build/${MIX_ENV}/lib/evision/priv/evision.so"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/priv/evision.so"

# delete evision related CMake build caches.
rm -rf "_build/${MIX_ENV}/lib/evision/cmake_evision"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/cmake_evision"

Current Status

{:ok, gray_mat} = OpenCV.imread("/path/to/img.png", flags: OpenCV.cv_imread_grayscale)
{:ok, gray_blur_mat} = OpenCV.blur(gray_mat, [10,10], anchor: [1,1])
{:ok, colour_mat} = OpenCV.imread("/path/to/img.png")
{:ok, colour_blur_mat} = OpenCV.blur(colour_mat, [10,10], anchor: [1,1])
:ok = OpenCV.imwrite("/path/to/img-gray-and-blur.png", gray_blur_mat)
:ok = OpenCV.imwrite("/path/to/img-colour-and-blur.png", colour_blur_mat)

{:ok, cap} = OpenCV.VideoCapture.videocapture(0)
{:ok, cap_mat} = OpenCV.VideoCapture.read(cap)
:ok = OpenCV.imwrite("/path/exists/capture-mat.png", cap_mat)
:error = OpenCV.imwrite("/path/not/exists/capture-mat.png", cap_mat)

Todo

  • Update .py files in py_src so that they output header files for Erlang bindings.

  • Automatically generate erl_cv_nif.ex.

  • Automatically generate opencv_*.ex files using Python.

  • Automatically convert enum constants in C++ to "constants" in Elixir

  • When a C++ function's return value's type is bool, map true to :ok and false to :error.

    # not this
{:ok, true} = OpenCV.imwrite("/path/to/save.png", mat, [])
# but this
:ok = OpenCV.imwrite("/path/to/save.png", mat, [])

  • Make optional parameters truly optional.

    # not this
{:ok, cap} = OpenCV.VideoCapture.videocapture(0, [])
# but this
{:ok, cap} = OpenCV.VideoCapture.videocapture(0)
# this also changes the above example to
:ok = OpenCV.imwrite("/path/to/save.png", mat)

  • Nerves support (rpi4 for now).

  • Add OpenCV.Mat module.

    {:ok, type} = OpenCV.Mat.type(mat)
{:ok, {:u, 8}}

{:ok, {height, weight, channel}} = OpenCV.Mat.shape(mat)
{:ok, {1080, 1920, 3}}
{:ok, {height, weight}} = OpenCV.Mat.shape(gray_mat)
{:ok, {1080, 1920}}

{:ok, bin_data} = OpenCV.Mat.to_binary(mat)
{:ok, << ... binary data ... >>}

  • Add support for Nx.

    nx_tensor = OpenCV.Mat.to_nx(mat)
#Nx.Tensor<
   u8[1080][1920][3]
   [[ ... pixel data ... ]]
>
{:error, reason} = OpenCV.Mat.to_nx(invalid_mat)

  • Edit config/config.exs to enable/disable OpenCV modules and image coders.

  • Add tests.

How does this work?

  1. This project will first pull OpenCV source code from git (as git submodules).

  2. Inside the OpenCV project, there is an opencv-python module, 3rd_party/opencv/modules/python. If the opencv-python module is enabled,

    cmake ...
    -D PYTHON3_EXECUTABLE=$(PYTHON3_EXECUTABLE) \
    ...

    It will generate files for opencv-python bindings in cmake build dir, cmake_build_dir/modules/python_bindings_generator.

    We are interested in the headers.txt file as it tells us which headers should we parse (this header list changes based on the enabled modules).

    We also need to check another file, pyopencv_custom_headers.h. This file includes pyopencv compatible headers from modules that need special handlings to enable interactions with Python. We will talk about this later.

  3. Originally, the headers.txt file will be passed to 3rd_party/opencv/modules/python/src2/gen2.py and that Python script will then generate pyopencv_*.h in cmake_build_dir/modules/python_bindings_generator. Here we copy that Python script and modify it so that it outputs c_src/evision_*.h files which use c_src/erlcompat.hpp and c_src/nif_utils.hpp to make everything compatible with Erlang.

  4. c_src/opencv.cpp includes almost all specialised and generic evision_to and evision_from functions. They are used for making conversions between Erlang and C++. Some conversion functions are defined in module custom headers.

  5. This is where we need to make some changes to pyopencv_custom_headers.h. We first copy it to c_src/evision_custom_headers.h and copy every file it includes to c_src/evision_custom_headers/. Then we make corresponding changes to c_src/evision_custom_headers/*.hpp files so that these types can be converted from and to Erlang terms. The header include path in c_src/evision_custom_headers.h should be changed correspondingly.

  6. However, it is hard to do step 5 automatically. We can try to create a PR which puts these changed files to the original OpenCV repo's {module_name}/mics/erlang/ directory. Now we just manually save them in c_src/evision_custom_headers. Note that step 5 and 6 are done manually, calling py_src/gen2.py will not have effect on c_src/evision_custom_headers.h and *.hpp files in c_src/evision_custom_headers.

  7. Another catch is that, while function overloading is easy in C++ and optional arguments is simple in Python, they are not quite friendly to Erlang/Elixir. There is basically no function overloading in Erlang/Elixir. Although Erlang/Elixir support optional argument (default argument), it also affects the function's arity and that can be very tricky to deal with. For example,

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts \\ []), do: :nil
  def open(self, filename), do: :nil
  # ... other functions ...
end

    In this case, def open(self, camera_index, opts \\ []), do: :nil will define open/3 and open/2 at the same time. This will cause conflicts with def open(self, filename), do: :nil which defines open/2.

    So we cannot use default arguments. Now, say we have

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts), do: :nil
  def open(self, filename, opt), do: :nil
end

    Function overloading in C++ is relatively simple as compiler does that for us. In this project, we have to do this ourselves in the Erlang/Elixir way. For the example above, we can use guards.

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts) when is_integer(camera_index) do
    # do something
  end
  def open(self, filename, opt) when is_binary(filename) do
    # do something
  end
end

    But there are some cases we cannot distinguish the argument type in Erlang/Elixir becauase they are resources (instance of a certain C++ class).

    defmodule OpenCV.SomeModule do
  # @param mat: Mat
  def func_name(mat) do
    # do something
  end

  # @param mat: UMat
  def func_name(mat) do
    # do something
  end
end

    In such cases, we only keep one definition. The overloading will be done in c_src/opencv.cpp (by evision_to).

  8. Enum handling. Originally, PythonWrapperGenerator.add_const in py_src/gen2.py will be used to handle those enum constants. They will be saved to a map with the enum's string representation as the key, and, of course, enum's value as the value. In Python, when a user uses the enum, say cv2.COLOR_RGB2BGR, it will perform a dynamic lookup which ends up calling corresponding evision_[to|from]. evision_[to|from] will take the responsibility to convert between the enum's string representation and its value. Although in Erlang/Elixir we do have the ability to both create atoms and do the similar lookups dynamically, the problem is that, if an enum is used as one of the arguments in a C++ function, it may be written as void func(int enum) instead of void func(ENUM_TYPE_NAME enum). However, to distinguish between overloaded functions, some types (int, bool, string, char, vector) will be used for guards. For example, void func(int enum) will be translated to def func(enum) when is_integer(enum), do: :nil. Adding these guardians help us to make some differences amongst overloaded functions in step 7. However, that prevents us froming passing an atom to def func(enum) when is_integer(enum), do: :nil. Technically, we can add one more variant def func(enum) when is_atom(enum), do: :nil for this specific example, but there are tons of functions has one or more ints as their input arguments, which means the number of variants in Erlang will increase expoentially (for each int in a C++ function, it can be either a real int or an enum). Another way is just allow it to be either an integer or an atom:

    def func(enum) when is_integer(enum) or is_atom(enum) do
  :nil
end

    But in this way, atoms are created on-the-fly, users cannot get code completion feature for enums from their IDE. But, finally, we have a good news that, in Erlang/Elixir, when a function has zero arguments, you can write its name without explictly calling it, i.e.,

    defmodule M do
  def enum_name(), do: 1
end

1 = M.enum_name
1 = M.enum_name()

    So, in this project, every enum is actually transformed to a function that has zero input arguments.

How do I make it compatible with more OpenCV modules?

Because of the reason in step 6, when you enable more modules, if that module has specialised custom header for python bindings, the custom headers will be added to cmake_build_dir/modules/python_bindings_generator/pyopencv_custom_headers.h. Then you can manually copy corresponding specialised custom headers to c_src/evision_custom_headers and modify these conversion functions in them.

Acknowledgements

  • gen2.py, hdr_parser.py and c_src/erlcompat.hpp were directly copied from the python module in the OpenCV repo. Changes applied.
  • Makefile, CMakeLists.txt and c_src/nif_utils.hpp were also copied from the torchx module in the elixir-nx repo. Minor changes applied.
Comments
  • Proposal: bridging with Nx

    Proposal: bridging with Nx

    I'm developing Excv, a bridge between Nx and OpenCV: https://github.com/zeam-vm/excv

    I hope that evision will be integrated with such bridging. I'd like to contribute to evision if you wish so. Would you please inform me where the #Reference structure includingMat?

    opened by zacky1972 14
  • Share image data between Vix and eVision (and Nx)

    Share image data between Vix and eVision (and Nx)

    I am the author of the Image library that is only possible because of @akash-akya's Vix which is a binding to libvips. Given Vix and eVision have similar problem domains, and very similar matrix implementations, I would like to be able to share image data between the two libraries. This would allow Image (and of course any other library) to leverage the capabilities of both Vix and eVision (and very desirably Nx) in a very memory efficient and performant manner.

    Objectives

    1. Share image data without copying the matrix (I think a shared interpretation is possible) - but need to make sure we know who has memory ownership and who is managing the memory (and I'm not good at this part).
    2. Having a way to agree and share data formats (ie a common lexicon to refer to image data type and matrix shape - perhaps using the Nx terminology?)
    3. Act as a model of how to do the same integration with Nx. I know eVision has Nx integration but it appears to be based upon converting to and from Elixir binaries - which I'd prefer not to do since lots of NIF copying would contradict the memory efficiency and performance objectives.

    I'm looking for guidance, suggestions and ideas - and hopefully someone motivated to tackle this because its a bit beyond my competence. :-)

    Given the intent to also provide integration with Nx, any comments from @josevalim would also be very welcome.

    enhancement 
    opened by kipcole9 13
  • Import Nx tensor that is in {width, height, channels} shape with RGB data format

    Import Nx tensor that is in {width, height, channels} shape with RGB data format

    Cocoa, I have tried everything I can think of to take an Nx tensor that is of the shape {width, height, channels} that contains RGB format data, and convert it in eVision to {height, width, channels} in BGR format.

    Any chance I might ask for your advice and recommendations?

    What I've tried

    Here is what I have tried which is, I think, the required process but the saved image is definitely not what is expected!

    tensor = File.read!("path_to/color_checker.etf") |> :erlang.binary_to_term()
{:ok, mat} = Evision.Nx.to_mat(tensor)
{:ok, transposed} = Evision.Mat.transpose(mat, [1, 0, 2])
{:ok, bgr} = Evision.cvtColor(transposed, Evision.cv_COLOR_RGB2BGR())
Evision.imwrite "some_path/color_checker.jpg", bgr

    I have followed the converse process in Image and the results line up with expectations. Which is not too surprising since Image expects data in {width, height, channels} and RGB format. I mention this just to note that I have verified that the tensor does represent the underlying example image.

    Artifacts

    The image converted to a tensor and stored with :erlang.term_to_binary/1 and zipped: color_checker.etf.zip

    The original image: color_checker

    opened by kipcole9 10
  • cropping of image using ROI not working

    cropping of image using ROI not working

    Please refer to the below images to understand about the issue. This is a function used to get image: Screenshot from 2022-10-10 10-50-12 This is the error window: Screenshot from 2022-10-10 10-52-21 Its working in python for reference: Screenshot from 2022-10-10 10-52-55

    opened by navrkr 9
  • Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

    Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

    Hi,

    I did the following steps:

    mix new evision_test
cd evision_test

    Then, I added evision to mix.exs as follows:

      defp deps do
    [
      {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
    ]
  end

    Then, I did the following steps:

    mix deps.get
mix compile

    Then, I got the following error:

    ==> evision
[/Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip or
        /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.zip, and cannot find /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.ZIP, period.
make: *** [/Users/zacky/evision_test/deps/evision/3rd_party/opencv/opencv-4.5.5/modules/core/include/opencv2/core/utils/configuration.private.hpp] Error 9
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    Then, I did the following steps:

    mix deps.compile evision

    Then, I got the following fatal error:

    ==> evision
-- The CXX compiler identification is AppleClang 13.0.0.13000029
-- The C compiler identification is AppleClang 13.0.0.13000029
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
(snip)
-- General configuration for OpenCV 4.5.5 =====================================
--   Version control:               ca737c9
-- 
--   Platform:
--     Timestamp:                   2022-01-26T09:38:53Z
--     Host:                        Darwin 21.2.0 arm64
--     CMake:                       3.22.1
--     CMake generator:             Unix Makefiles
--     CMake build tool:            /usr/bin/make
--     Configuration:               RELEASE
-- 
--   CPU/HW features:
--     Baseline:                    NEON FP16
-- 
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /Library/Developer/CommandLineTools/usr/bin/c++  (ver 13.0.0.13000029)
--     C++ flags (Release):         -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /Library/Developer/CommandLineTools/usr/bin/cc
--     C flags (Release):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,-dead_strip  
--     Linker flags (Debug):        -Wl,-dead_strip  
--     ccache:                      NO
--     Precompiled headers:         NO
--     Extra dependencies:
--     3rdparty dependencies:
-- 
--   OpenCV modules:
--     To be built:                 calib3d core dnn features2d flann highgui imgcodecs imgproc ml objdetect photo python2 stitching ts video videoio
--     Disabled:                    gapi world
--     Disabled by dependency:      -
--     Unavailable:                 java python3
--     Applications:                perf_tests apps
--     Documentation:               NO
--     Non-free algorithms:         NO
-- 
--   GUI:                           COCOA
--     Cocoa:                       YES
--     VTK support:                 NO
-- 
--   Media I/O: 
--     ZLib:                        zlib (ver 1.2.11)
--     JPEG:                        build-libjpeg-turbo (ver 2.1.2-62)
--     WEBP:                        build (ver encoder: 0x020f)
--     PNG:                         libpng (ver 1.6.37)
--     TIFF:                        libtiff (ver 42 / 4.2.0)
--     JPEG 2000:                   build (ver 2.4.0)
--     OpenEXR:                     OpenEXR::OpenEXR (ver 3.1.3)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
-- 
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                YES (4.0.0)
--     GStreamer:                   NO
--     AVFoundation:                YES
-- 
--   Parallel framework:            GCD
-- 
--   Trace:                         YES (with Intel ITT)
-- 
--   Other third-party libraries:
--     Lapack:                      YES (/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/Accelerate.framework -lm -ldl)
--     Eigen:                       YES (ver 3.4.0)
--     Custom HAL:                  YES (carotene (ver 0.0.1))
--     Protobuf:                    build (3.19.1)
-- 
--   OpenCL:                        YES (no extra features)
--     Include path:                NO
--     Link libraries:              -framework OpenCL
-- 
--   Python 2:
--     Interpreter:                 /usr/bin/python2.7 (ver 2.7.18)
--     Libraries:                   /usr/lib/libpython2.7.dylib (ver 2.7.18)
--     numpy:                       /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include (ver 1.8.0rc1)
--     install path:                lib/python2.7/site-packages/cv2/python-2.7
-- 
--   Python (for build):            /usr/bin/python2.7
-- 
--   Java:                          
--     ant:                         NO
--     JNI:                         NO
--     Java wrappers:               NO
--     Java tests:                  NO
-- 
--   Install to:                    /Users/zacky/evision_test/_build/dev/lib/evision/priv
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5
[  0%] Building C object 3rdparty/ittnotify/CMakeFiles/ittnotify.dir/src/ittnotify/ittnotify_static.c.o
[  0%] Generate opencv4.pc
[  0%] Building C object 3rdparty/zlib/CMakeFiles/zlib.dir/adler32.c.o
[  0%] Building C object 3rdparty/openjpeg/openjp2/CMakeFiles/libopenjp2.dir/thread.c.o
(snip)
[ 72%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/vkcom/vulkan/vk_loader.cpp.o
[ 72%] Linking CXX shared library ../../lib/libopencv_dnn.dylib
[ 72%] Built target opencv_dnn
make[1]: *** [all] Error 2
make: *** [/Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 2
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    My environment is as follows:

    $ uname -a 
Darwin zackym1air.local 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:29:10 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T8101 arm64
$ elixir -v
Erlang/OTP 24 [erts-12.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Elixir 1.13.2 (compiled with Erlang/OTP 24)

    Thank you.

    opened by zacky1972 9
  • Evision.DNN.DetectionModel can't setInputParams at YOLOv4

    Evision.DNN.DetectionModel can't setInputParams at YOLOv4

    I tried to use YOLOv4 at Evision.DNN.DetectionModel

    dog = "yolov4/dog.jpg"
weights = "yolov4/yolov4.weights"
config = "yolov4/yolov4.cfg"
classes = "yolov4/coco.names"
mat = Evision.imread(dog)

Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.DetectionModel.detect(mat)

    return error

    {:error,
 "OpenCV(4.6.0) /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/dnn/src/model.cpp:98: error: (-201:Incorrect size of input array) Input size not specified in function 'processFrame'\n"}

    i guess need such as python code

    model.setInputParams(1.0, (416, 416), (0.0, 0.0, 0.0), true, false)

    but

    Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams(scale: 1.0, size: {416, 416}, mean: {0, 0, 0}, swapRB: true, crop: false)

    ** (UndefinedFunctionError) function Evision.DNN.Model.setInputParams/2 is undefined or private. Did you mean:

    Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams({416, 416})

    {:error, "cannot get `cv::dnn::Model` from `self`: mismatched type or invalid resource?"}

    How about the following solution? 1 Convert the loaded model to Evision.DNN.Model 2 Enable setInputParams in DetectionModel as well

    Evision.DNN.readFromDarknet load and detect are success but, I wan't to impl post process :-(

    Enviroments evision 0.1.16 elixir 1.14.1 elrang 25.0.1 os macOS 12.6

    opened by thehaigo 8
  • small doubt regarding contours in evision

    small doubt regarding contours in evision

    i get the values of findContours after getting is their any method to seperate contours using in built function like in python imutil.grabcontours

    the code is here

    `def grab_contours(cnts):
    # if the length the contours tuple returned by cv2.findContours
    # is 2 then we are using either OpenCV v2.4, v4-beta, or
    # v4-official
    if len(cnts) == 2:
        cnts = cnts[0]

    # if the length of the contours tuple is 3 then we are using
    # either OpenCV v3, v4-pre, or v4-alpha
    elif len(cnts) == 3:
        cnts = cnts[1]

    # otherwise OpenCV has changed their cv2.findContours return
    # signature yet again and I have no idea WTH is going on
    else:
        raise Exception(("Contours tuple must have length 2 or 3, "
            "otherwise OpenCV changed their cv2.findContours return "
            "signature yet again. Refer to OpenCV's documentation "
            "in that case"))

    # return the actual contours array
    return cnts`

    could please help us sir @cocoa-xu @josevalim

    opened by Jagan3534 8
  • An error occur when compiling on Jetson (Linux, aarch64)

    An error occur when compiling on Jetson (Linux, aarch64)

    Hi,

    I compiled evision on NVIDIA Jetson AGX Xavier (Ubuntu 18.04 Linux Kernel 4.9.201-tegra, aarch64), but the following error occurred:

    ==> evision
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   117    0   117    0     0    393      0 --:--:-- --:--:-- --:--:--   393
100 89.8M    0 89.8M    0     0  9401k      0 --:--:--  0:00:09 --:--:-- 14.4M
CMake Error: The source directory "/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/BUILD_OPENEXR=ON" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
Makefile:96: recipe for target '/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt' failed
make: *** [/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 1
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. If you are using
Ubuntu or any other Debian-based system, install the packages
"build-essential". Also install "erlang-dev" package if not
included in your Erlang/OTP version. If you're on Fedora, run
"dnf group install 'Development Tools'".

    Are Linux and aarch64 not supported?

    opened by zacky1972 8
  • imshow function is broken in pre-compiled library.

    imshow function is broken in pre-compiled library.

    I recently updated from elixir 1.0.0-dev to 1.0.9 version of Evision and everything is great till now, just tried getting the image as output and this error popped (Note: All the issue is with cvwait and imshow doesn't work without cvwait).

    IMG_20221011_070312_455

    opened by navrkr 7
  • sorry for what is did . we have a small problem in sort sir

    sorry for what is did . we have a small problem in sort sir

    I apologise if i sound rude, i did not know english that well may be this is why i sounds rude.

    here same we got the error in the elixir in python we wrote like this in elixir we tried to give but in sort 2 nd parameter is doubted sir in Evision hexdoxs we tried to see the example but in new version they dont show contours = sorted(contours, key=cv2.contourArea, reverse=True)

    could you please help us

    opened by Jagan3534 6
  • Can't round trip QRcode encode/decode

    Can't round trip QRcode encode/decode

    QRcode detection is working well (as far as I can test) but when I generate a QRcode and attempt to detect it I am not having any luck. For example:

    iex> qrcode = Evision.QRCodeEncoder.encode Evision.QRCodeEncoder.create(), "This is a string"           
%Evision.Mat{
  channels: 1,
  dims: 2,
  type: {:u, 8},
  raw_type: 0,
  shape: {25, 25},
  ref: #Reference<0.1002129465.3225026580.34308>
}
iex> Evision.QRCodeDetector.detectAndDecode Evision.QRCodeDetector.qrCodeDetector(), qrcode             
{"",
 %Evision.Mat{
   channels: 2,
   dims: 2,
   type: {:f, 32},
   raw_type: 13,
   shape: {1, 4, 2},
   ref: #Reference<0.1002129465.3225026580.34310>
 }, {:error, "empty matrix"}}
    1. The encoded image is very small, {25, 25} and I don't actually know how - even if its possible - to define parameters for the encoder to make the image larger.
    2. I can see there is a module Evision.QRCodeEncoder.Params but I'm not having any success try to define a param structure. For example:
    iex> Evision.QRCodeEncoder.Params.get_mode Evision.QRCodeEncoder.Params.qrcodeencoder_params()
zsh: segmentation fault  iex -S mix

    Thanks for your patience @cocoa-xu while I work out have to navigate the wonderful world of OpenCV/eVision - help and guidance much appreciated.

    opened by kipcole9 6
  • [Feature] opencv-contrib modules

    [Feature] opencv-contrib modules

    I plan to add support for modules in opencv_contrib gradually. Of course, this would bring an inevitable question: not everyone will use modules in opencv_contrib, so including them in the precompiled binaries will increase the footprint.

    Will it be feasible to have an :evision_contrib library in Elixir (like its python counterpart, where we have opencv-python and opencv-python-contrib)?

    Modules (part 1)

    • [x] barcode
    • [x] dnn_superres
    • [x] face
    • [x] img_hash
    • [x] plot
    • [x] quality
    • [x] rapid
    • [x] sfm
    • [x] shape
    • [x] structured_light
    • [x] text
    • [x] tracking
    • [x] xfeatures2d
    • [x] ximgproc
    • [x] xphoto

    Modules (part 2)

    • [x] aruco
    • [x] bgsegm
    • [x] bioinspired
    • [x] hfs
    • [x] line_descriptor
    • [x] mcc
    • [x] reg
    • [x] rgbd
    • [x] saliency
    • [x] stereo
    • [x] surface_matching
    • [x] wechat_qrcode

    Modules (part 3)

    • [x] cudaarithm
    • [x] cudabgsegm
    • [x] cudacodec
    • [x] cudafeatures2d
    • [x] cudafilters
    • [x] cudaimgproc
    • [x] cudalegacy
    • [x] cudaobjdetect
    • [x] cudaoptflow
    • [x] cudastereo
    • [x] cudawarping
    • [x] cudev

    todo

    • [x] Check module name and struct name
    • [ ] Check _Params classes
    • [ ] Examples
    • [ ] Tests
    enhancement 
    opened by cocoa-xu 2
  • [Feature-GUI] Implementing `Evision.Wx`

    [Feature-GUI] Implementing `Evision.Wx`

    As mentioned in #103:

    Even compiling OpenCV with highgui module, sometimes the Evision.HighGui.imshow function does not really work as expected (in evision): the window(s) it opened will not respond to any GUI events until calling Evision.HighGui.waitKey again.

    To make it as good as one would expect it to be, like how it is in C++/Python, perhaps it would require one to invest a few weeks of time for it. But I'm focusing on other aspects of this library right now, also I'm hesitating to do it because I don't know if this would worth that amount time.

    Also, as :wx has official support from Erlang, and it's well tested, and one can even build a relatively complex GUI app with :wx if they want.

    This module will try to make itself easy to use. Meanwhile, the Evision.HighGui module will stay in the source code even after Evision.Wx is completed.

    Here is a non-exhaustive list of desired/prospective functions in Evision.Wx. All of them, including the type and the number of their input arguments, are subject to changes in the future, or will not be implemented if the desired functionality is partially or wholly impossible to achieve in Erlang/Elixir with :wx.

    Functions available in OpenCV's highgui module can be found here.

    Any contributions or suggestions (either about Evision.Wx or Evision.HighGui) are more than welcomed - this also applies to the whole project. :)

    enhancement 
    opened by cocoa-xu 0
  • Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

    Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

    cv::Mat (and its variants) is the OpenCV's implementation of multi-dimensional array, so technically, it can be used as a backend for the Nx (numerical-elixir).

    opened by cocoa-xu 8
  • OpenCV gapi module

    OpenCV gapi module

    Based on the contents of pyopencv_gapi.hpp, it seems that the binding code will need to call an Elixir function and fetch the result.

    This perhaps could be done if we send related ERL_NIF_TERMs to a separate Erlang thread and send the result back. But it's not easy to properly design and write the whole thing.

    Maybe it's not worth the effort... I probably won't port this module anytime soon.

    This module is highly likely to be excluded from evision 0.1.0.

    opened by cocoa-xu 0
Releases(v0.1.25)
Owner
Cocoa
Ph.D student @ UofG
Cocoa
GitHub Repository
A tensorflow implementation of EAST text detector

EAST: An Efficient and Accurate Scene Text Detector Introduction This is a tensorflow re-implementation of EAST: An Efficient and Accurate Scene Text

2.9k Jan 02, 2023
This is the open source implementation of the ICLR2022 paper "StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis"

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image

Meta Research 840 Dec 26, 2022
PyQT5 app that colorize black & white pictures using CNN(use pre-trained model which was made with OpenCV)

About PyQT5 app that colorize black & white pictures using CNN(use pre-trained model which was made with OpenCV) Colorizor Приложение для проекта Yand

1 Apr 04, 2022
scantailor - Scan Tailor is an interactive post-processing tool for scanned pages.

Scan Tailor - scantailor.org This project is no longer maintained, and has not been maintained for a while. About Scan Tailor is an interactive post-p

1.5k Dec 28, 2022
A machine learning software for extracting information from scholarly documents

GROBID GROBID documentation Visit the GROBID documentation for more detailed information. Summary GROBID (or Grobid, but not GroBid nor GroBiD) means

Patrice Lopez 1.9k Jan 08, 2023
OCR powered screen-capture tool to capture information instead of images

NormCap OCR powered screen-capture tool to capture information instead of images. Links: Repo | PyPi | Releases | Changelog | FAQs Content: Quickstart

575 Dec 31, 2022
Ddddocr - 通用验证码识别OCR pypi版

带带弟弟OCR通用验证码识别SDK免费开源版 今天ddddocr又更新啦！ 当前版本为1.3.1 想必很多做验证码的新手，一定头疼碰到点选类型的图像，做样本费时

Sml2h3 4.4k Dec 31, 2022
A python scripts that uses 3 different feature extraction methods such as SIFT, SURF and ORB to find a book in a video clip and project trailer of a movie based on that book, on to it.

A python scripts that uses 3 different feature extraction methods such as SIFT, SURF and ORB to find a book in a video clip and project trailer of a movie based on that book, on to it.

tooraj taraz 3 Feb 10, 2022
Automatically download multiple papers by keywords in CVPR

CVFPaperHelper Automatically download multiple papers by keywords in CVPR Install mkdir PapersToRead cd PaperToRead pip install requests tqdm git clon

46 Jun 08, 2022
Isearch (OSINT) 🔎 Face recognition reverse image search on Instagram profile feed photos.

isearch is an OSINT tool on Instagram. Offers a face recognition reverse image search on Instagram profile feed photos.

Malek salem 20 Oct 25, 2022
Play the Namibian game of Owela against a terrible AI. Built using Django and htmx.

Owela Club A Django project for playing the Namibian game of Owela against a dumb AI. Built following the rules described on the Mancala World wiki pa

Adam Johnson 18 Jun 01, 2022
Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.

EasyOCR Ready-to-use OCR with 80+ languages supported including Chinese, Japanese, Korean and Thai. What's new 1 February 2021 - Version 1.2.3 Add set

Jaided AI 16.7k Jan 03, 2023
Let's explore how we can extract text from forms

Form Segmentation Let's explore how we can extract text from any forms / scanned pages. Objectives The goal is to find an algorithm that can extract t

Philip Doxakis 42 Jun 05, 2022
Implementation of our paper 'PixelLink: Detecting Scene Text via Instance Segmentation' in AAAI2018

Code for the AAAI18 paper PixelLink: Detecting Scene Text via Instance Segmentation, by Dan Deng, Haifeng Liu, Xuelong Li, and Deng Cai. Contributions

758 Dec 22, 2022
Dirty, ugly, and hopefully useful OCR of Facebook Papers docs released by Gizmodo

Quick and Dirty OCR of Facebook Papers Gizmodo has been working through the Facebook Papers and releasing the docs that they process and review. As lu

Bill Fitzgerald 2 Oct 28, 2021
Tool which allow you to detect and translate text.

Text detection and recognition This repository contains tool which allow to detect region with text and translate it one by one. Description Two pretr

Damian Panek 176 Nov 28, 2022
Morphological edge detection or object's boundary detection using erosion and dialation in OpenCV python

Morphologycal-edge-detection-using-erosion-and-dialation the task is to detect object boundary using erosion or dialation . Here, use the kernel or st

Tamzid hasan 3 Nov 25, 2022
Official implementation of Character Region Awareness for Text Detection (CRAFT)

CRAFT: Character-Region Awareness For Text detection Official Pytorch implementation of CRAFT text detector | Paper | Pretrained Model | Supplementary

Clova AI Research 2.5k Jan 03, 2023
Learning Camera Localization via Dense Scene Matching, CVPR2021

This repository contains code of our CVPR 2021 paper - "Learning Camera Localization via Dense Scene Matching" by Shitao Tang, Chengzhou Tang, Rui Hua

tangshitao 65 Dec 01, 2022
PSENet - Shape Robust Text Detection with Progressive Scale Expansion Network.

News Python3 implementations of PSENet [1], PAN [2] and PAN++ [3] are released at https://github.com/whai362/pan_pp.pytorch. [1] W. Wang, E. Xie, X. L

1.1k Dec 24, 2022