FaceTracker (DX11.Texture 2d)

So I found this library: https://github.com/ShiqiYu/libfacedetection And it was super easy to use. So after a wrapper class and a handling node here are the results from vvvv with any DX11 texture:

title

Trololo at 60-70 fps with multiview-reinforce engine.

crowd crowd

Crowd footage at 10-20 fps with multiview engine. It tracks as many faces as much you throw at it.

This library is fast but far from accurate. But I made the wrapper class in a way that in the future it can be easily extended with other engines, so if anyone finds robust and fast face trackers, throw that at me and I’ll implement it if they don’t cost a fortune.

Available in mp.dx http://vvvvpm.github.io/#mp.dx Enjoy!

Comments:

Comments are no longer accepted for this post.

Lrinke

24.01.2018 - 15:35

that’s f*** awesome :D next project yolo object detection? https://pjreddie.com/darknet/yolo/

microdee

24.01.2018 - 21:46

it’s up and running now in mp.dx: http://vvvvpm.github.io/#mp.dx check out the FaceTracker patch in girlpower

ggml

06.05.2018 - 23:38

microdeeuser is this implementation available for try ?

microdee

07.05.2018 - 03:05

it’s available in mp.dx, https://vvvvpm.github.io/#mp.dx/latest

tobyk

22.05.2018 - 19:31

Hi @Microdee. Having trouble with this one if you’ve got a tick…

Under vvvv_50beta36_x64 I get a red FaceTrackerContent node. The error pin on the FaceTracker node says

An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
   at mp.dx.Tracking.FaceTrackerContext.facedetect_frontal_surveillance(Byte* result_buffer, Byte* gray_image_data, Int32 width, Int32 height, Int32 step, Single scale, Int32 min_neighbors, Int32 min_object_width, Int32 max_object_width, Int32 doLandmark)
   at mp.dx.dx11.Nodes.TextureFaceTrackerNode.Evaluate(Int32 SpreadMax) in C:\projects\mp-dx\DX11Utils\TextureFaceTrackerNode.cs:line 150

Installed via VPM github.latest.vpack Tried with latest DX11 pack (1.3 released a few days ago) and with the DX11 pack that VPM installs.

microdee

24.05.2018 - 15:25

hi! thanks I will check if the necessary dll’s are copied properly later, I have a suspicion it doesn’t do that

rogalag

25.05.2018 - 20:00

Hi @Microdee: Something’s wrong downloading your vpm pack . I’m using latest vpm 1.2.8 Maybe you can provide alternative “classic style” download link

microdee

25.05.2018 - 23:22

that there looks like an edge case. check your SSH situation, if you use a proxy or a vpn make sure they don’t fiddle with secure channels. You can have “classic style downloading” but you’ll need to download and extract 5 zip files.

https://github.com/microdee/mp.essentials/releases/latest https://github.com/microdee/mp.dx/releases/latest https://github.com/microdee/mp.fxh/archive/master.zip https://github.com/velcrome/vvvv-Message/releases/tag/2.9.10

directx11-nodesbeta-contribution

extract all to packs folder. also mp.* packs require .NET 4.7+ so make sure your windows 7 is up to date (so turn updates back on if you turned them off).

I still didn’t have time to fix the face tracker dll automatic deployment, if you can’t wait though download this https://github.com/microdee/mp.dx/blob/master/DX11Utils.Native/dependencies/libfacedetect.dll and put it next to DX11Utils.dll inside the plugins folder of mp.dx. x64 only.

rogalag

26.05.2018 - 14:43

Thanks a lot @microdee, I will try to collect pack , hope it’ll work…

rogalag

27.05.2018 - 19:23

@microdee Thank you!. Facetracker working perfectly! G.

vj shanti

11.08.2018 - 12:57

Hi there! How can I instal this contribution?

Cheers

kine

02.07.2019 - 13:44

Hi @microdee! Thanks for this contribution! I am using VVVV50beta38.1. Just tried to install it through VPM, but when I open FaceTracker girlpower patch I have red nodes issues (see screenshot). Can you help me solving the problem? Thanks! It appears that the DX11Utils are missing. I looked into pack folders and the DX11Utils was in a different folder respect to the one where the nodes are searching for. So I tried to copy all the DX11Utils in the correct path but the nodes are still red. I attach both TTYlog and the exception error message I receive when I try to open Facetracker girlpower patch with /showexeptions.

System.IO.FileNotFoundException: Could not load file or assembly ‘mp.pddn, Version=1.3.2.0, Culture=neutral, PublicKeyToken=null’ or one of its dependencies. The system cannot find the file specified. File name: ‘mp.pddn, Version=1.3.2.0, Culture=neutral, PublicKeyToken=null’ at System.Reflection.RuntimeAssembly.GetExportedTypes(RuntimeAssembly assembly, ObjectHandleOnStack retTypes) at System.Reflection.RuntimeAssembly.GetExportedTypes() at VVVV.Hosting.Factories.DotNetPluginFactory.LoadNodeInfosFromFile(String filename, String sourcefilename, List1& nodeInfos, Boolean commitUpdates) at VVVV.Hosting.Factories.DotNetPluginFactory.LoadNodeInfos(String filename) at VVVV.Hosting.Factories.AbstractFileFactory1.ExtractNodeInfos(String filename, String arguments) at VVVV.Hosting.HDEHost.LoadNodeInfos(String filename, String arguments) at VVVV.Hosting.HDEHost.ExtractNodeInfos(String filename, String arguments, INodeInfo& result)

ggml

25.01.2018 - 02:03

nice!

microdee

02.07.2019 - 14:13

@vj shanti download this: https://github.com/vvvvpm/vpm/releases/tag/1.2.9 unzip / run once / follow on screen instructions clicky this: http://vvvvpm.github.io/#mp.dx then download this https://github.com/microdee/mp.dx/blob/master/DX11Utils.Native/dependencies/libfacedetect.dll and put it next to DX11Utils.dll inside the plugins folder of mp.dx. x64 only.

microdee

04.07.2019 - 13:51

it was missing mp.pddn.dll but vpm should have taken care of that. Please check if A: there’s an md.core folder and there’s an mp.pddn.dll inside that folder in your packs folder and B: your vvvv.exe.config file next to vvvv.exe (hidden by default) contains

<probing privatePath="lib\core;packs\md.core" />

kine

09.07.2019 - 11:34

Thanks @microdee this worked. My md.core folder was missing, I installed it with vpm and now everything is working.

Boris26

30.03.2022 - 14:42

Hey,

I’m trying to install this but VPM keeps throwing me this error :

Script thrown an Exception, contact the pack developer: The remote server returned an error: (403) Forbidden. at System.Net.WebClient.OpenRead(Uri address) at System.Net.WebClient.OpenRead(String address) at Submission#0.«Initialize»d__0.MoveNext()

I managed to install manually the mp.dx, mp.essentials, mp.fxh, vvvv.messages, but I don’t see the face track related nodes…

And I don’t have the md.core folder…

Any help ? :-)

bjoern

31.03.2022 - 10:38

@Boris26: deprecating-vpdb-(vvvvpm.github.io)

joreg

31.03.2022 - 11:18

a similar face mark detector is now also available with VL.OpenCV. when installed in beta or gamma, simply look for “HowTo Detect face landmarks” in the helpbrowser.

colorsound

25.01.2018 - 10:11

great ¡¡ thank you.

CeeYaa

25.01.2018 - 12:51

nize - thanks again, microdee

metrowave

26.01.2018 - 15:44

Cool, that’s funny…

microdee

27.01.2018 - 03:49

if anybody has red node issues install mp.dx through vpm again (only overwrite mp.dx), I forgot to set the build config back to release the first time, causing DLL issues for everyone without Visual Studio and without C++ development packs for it. Now it’s in release so should be fine with VC redist 2017

eps

20.02.2018 - 13:40

mucho gracias, ultra cool’n’useful

ggml

23.02.2018 - 12:55

hello; could this tracker aproximate the relative movement of faces ? (does it output ids?)

microdee

23.02.2018 - 15:52

nope, it’s only looking at single frame images, it can’t keep a track of faces in the time domain. however you can come up with something based on positions and searching for other consistencies accross frames.

FaceTracker (DX11.Texture 2d)

Comments:

Recent posts:

Filter :

Embedded YouTube Video

Embedded Vimeo Video

Contact

Imprint

Follow us

Sign up for our Newsletter