VL.OpenCV Release CandidateMarch 17, 2019 posted by: ravazquez Guest Post
As you know, efforts have been going for the last year and a half into bringing a computer vision nodeset to VL.
The goal was to incorporate as many of the features covered by the world renowned ImagePackbeta-contribution; contributed by elliotwoodsuser some years ago; while bringing in as many new features and improvements as we could.
In the winter of 2018, after setting a roadmap and having patched a decent initial nodeset, we happily announced the pre-release version of VL.OpenCV.
Since then, listening to your needs and constant feedback, we have tried to polish every corner, fix every bug, document every odd scenario, add plenty of demos and specially we tried to give you a clean, consistent and easy to use nodeset.
At this point in time, we are happy to announce that the goal has been nearly met. Most of the features available in the ImagePack made it into VL.OpenCV with the exception of Structured Light, Feature Detection, StereoCalibrate and some of the Contour detection functionality. At the same time, newer features such as YOLO, Aruco marker detection and others have been brought in for you to play with.
So what’s next? Even better documentation and loads of examples!
In the mean time, here is a summary of the new things that have been brought into the package in the last couple of months:
The new CvImage wrapper around OpenCV’s Mat type allows for some optimizations, specially when dealing with non-changing images.
CvImage allows nodes to run only once when the image has changed, significantly reducing CPU usage
Cache Region lovvvves CvImage!
Since it is now possible to detect if an image has changed, CvImage is a perfect candidate to benefit from Cache regions.
Cache regions can now make proper usage of image inputs and outputs
The Renderer was re-built from the ground up to improve usability and to fix bugs and issues. Title, Render Mode and Show Info features were added. Renderer also remembers its bounds on document reload.
New Renderer implementation introduces Titlepin, Renderer Modepin and Show Infopin pins
Histogram analysis has been added to VL.OpenCV. A useful tool in many scenarios.
Histograms allow you to analyze pixel value tendencies per channel
Homography and reverse homography are now available in VL.OpenCV.
Homography (vvvv used only for point IOBox)
Two new Stereo Matchers were added, these allow you to create a depth map from a set of stereo images. For more see the StereoDepth demo patch in VL.OpenCV.
Depth map obtained from a pair of stereo images
Serialization support was added for CvImage and Mat types, allowing you to use CvImage as part of more complex data structures which get serialized with no effort. This can be a heavy operation so make sure to trigger it when needed only.
For a hands on demonstration check out the Serialization VL demo that ships with VL.OpenCV.
As part of this final effort to clean everything even further and make the nodeset consistent and properly organized, we needed to rename and move a few things around which as you can imagine means the introduction of breaking changes. We understand this is an annoying thing to cope with, but this was basically the reason why we chose to keep this pack in a pre-release state until we felt confident with its structure and approach.
In summary yes, you will get red nodes when you upgrade your VL.OpenCV projects to the latest version, but in most cases it should be as easy as to double-click and find the node in its new category.
An exception to this are the nodes that got renamed, which we list below:
-Reader (Intrinsics) -> IntrinsicsReader -Reader (View Projection) -> ViewProjectionReader -Writer (Intrinsics) -> IntrinsicsWriter -Writer (View Projection) -> ViewProjectionWriter -GetByteArray -> ToByteArray -GetValues -> ToValues -GetValues (Custom) -> ToValues (Custom) -FiducialTracker -> FiducialDetector -Update (FaceRecognizer) -> FaceRecognizerUpdate -VideoFile (Append) -> VideoWriter (Append) -VideoFile - VideoPlayer -Merge -> Join
-Introduced a new wrapper for Mat called CvImage -Cache region additions to improve performance on non-changing Images -ImageReader re-implementation -Renderer re-implementation -VideoIn re-implementation -Reintroduced “Supported Formats” output pin on VideoIn node -Upgraded to VL.Core 0.95.37 -DirectX/OpenCV transformation documentation and cleanup -Added Blocking, Reactive and HoldLatestCopy versions of VideoIn node -Added Blocking and Reactive versions of ImageReader node -Reimplemented Extrinsics as a data type -Stabilized VideoFile Source node and moved from Experimental into Sources -Added Serialization support for Mat type -Added Serialization support for CvImage type
New in VL
Nodes -StereoBMMatcher -StereoSGBMMatcher -ToValues (Custom) -ToValues -GetChannel -Histogram -DrawHistogram -ToByteArray -Transform (Perspective)
Demos -DrawNodes -Homography -Histogram -Serialization -StereoSGBMMatcher
New in vvvv
Nodes -IntrinsicsToProjectionMatrix -ExtrinsicsToViewMatrix
Demos -Calibrate and Reproject -SolvePnP -StereoDepth
Remember that in order to use VL.OpenCV you first need to manually install it as explained here. Also, until we move away from pre-release you need to use the latest alpha builds.
We hope you find good use for this library in your computer vision projects and as always, please test and report.
Comments are no longer accepted for this post.
Looking forward to try this out!
ImagePack was nice, unfortunately also quite unstable sometimes. Really curious about the performance and stability characteristics of this little gem :)
p.s.: Finally a really good reason to jump onto the VL bandwagon!!!
@isdzaurov, yes, it is possible to do just that, we tested it and it worked. Be advised that this is an expensive operation so you might want to optimize performance. A couple of simple ways to do this would be to downscale the image before processing it and to make use of the AsyncLoop region to run this on its own thread.
Also, there are quite a lot of parameters to tweak and it can take some time before you get proper results based on your data. Let me know if you need any help.
@m9dfukc glad to hear you will be moving over to VL, you will not look back I am sure. As for stability and performance, stability seems to be quite good from our tests, as for performance, the underlying wrapper we are using (OpenCVSharp) does not have GPU support, so you do need to keep that in mind, but so far for us and other users trying the library out it has been good. If you are handling large images on high framerates you will very likely need to get creative.