Mate18 vs. Mate16 regression?

Post Reply
wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Sat May 04, 2019 8:58 am

I got my Odroid XU-4 as a better RaspberryPi for my IOT AI project.

I recently installed OpenVINO 2019R1 on the Pi3B, UX-4 Mate16, and Mate18. Raspbian is "supported, I had to hack the install scripts to get it running on the Odroid.

The OpenVINO 2019R1 allows building the C++ sample code. Python is my interest for the time being, and I had to "sideload" Python 3.5 on Mate18 to and install in a Python virtual environment get it to work on Mate18 since it ships with Python 3.6.

I tried building and running the C++ example from the installation instructions here: https://docs.openvinotoolkit.org/latest ... pbian.html

There appears to be a performance regression where Mate18 is significantly worse than the Mate16 system, remember this is C++ code, not Python. I get the following results from the sample code:
Odroid XU-4 Mate16
NCS: 8.22 fps
NCS2: 11.5 fps

Looks to be performance regression on Mate18 vs. Mate16!
Odroid XU-4 Mate18
NCS: 6.56 fps
NCS2: 8.36 fps

Raspberry Pi3B:
NCS: 6.93 fps
NCS2: 8.58 fps

Basically Mate18 is a bit worse than a Pi3B here!

I don't know what to make of this but the Mate16 c++ build was "clean" whereas the Mate18 and Raspbian builds threw multiple pages of warnings:

Code: Select all

Scanning dependencies of target object_detection_sample_ssd
[ 96%] Building CXX object object_detection_sample_ssd/CMakeFiles/object_detection_sample_ssd.dir/main.cpp.o
In file included from /usr/include/c++/7/algorithm:62:0,
                 from /home/odroid/inference_engine_vpu_arm/deployment_tools/inference_engine/samples/object_detection_sample_ssd/main.cpp:16:
/usr/include/c++/7/bits/stl_algo.h: In function ‘void std::__insertion_sort(_RandomAccessIterator, _RandomAccessIterator, _Compare) [with _RandomAccessIterator = __gnu_cxx::__normal_iterator<std::pair<std::__cxx11::basic_string<char>, InferenceEngine::InferenceEngineProfileInfo>*, std::vector<std::pair<std::__cxx11::basic_string<char>, InferenceEngine::InferenceEngineProfileInfo> > >; _Compare = __gnu_cxx::__ops::_Iter_comp_iter<perfCountersSorted(std::map<std::__cxx11::basic_string<char>, InferenceEngine::InferenceEngineProfileInfo>)::<lambda(const perfItem&, const perfItem&)> >]’:
/usr/include/c++/7/bits/stl_algo.h:1840:5: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<std::pair<std::__cxx11::basic_string<char>, InferenceEngine::InferenceEngineProfileInfo>*, std::vector<std::pair<std::__cxx11::basic_string<char>, InferenceEngine::InferenceEngineProfileInfo> > >’ changed in GCC 7.1
     __insertion_sort(_RandomAccessIterator __first,
     ^~~~~~~~~~~~~~~~
/usr/include/c++/7/bits/stl_algo.h:1840:5: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<std::pair<std:
These warnings appear to be the same on the Pi3B and the Mate18 Odroid.

My Mate16 is on an SD card, the Mate18 is the pre-loaded eMMC card, all systems are current with apt-get update, apt-get upgrade within the past two days.

Mate18:
(openvino) odroid@XU18:~/OpenVINO$ uname -a
Linux XU18 4.14.107-157 #1 SMP PREEMPT Thu Mar 21 09:59:50 -03 2019 armv7l armv7l armv7l GNU/Linux


Mate16:
odroid@XU16:~/OpenVINO$ uname -a
Linux XU16 4.14.111-139 #1 SMP PREEMPT Tue Apr 16 17:31:00 UTC 2019 armv7l armv7l armv7l GNU/Linux

Pi3B:
pi@ov2019R1:~/OpenVINO $ uname -a
Linux ov2019R1 4.14.98-v7+ #1200 SMP Tue Feb 12 20:27:48 GMT 2019 armv7l GNU/Linux


On my i3 and i5 laptops I've found Mate18 to be inferior to to Mate16 but nothing so dramatic and easily illustrated.

User avatar
odroid
Site Admin
Posts: 32677
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 209 times
Been thanked: 364 times
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by odroid » Tue May 07, 2019 12:19 pm

Is "OpenVINO" using the OpenGL-ES API or a generic OpenGL?

More modern gcc might have more tight c++ syntax parsing and it might cause the warning message while compiling the code.

wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Thu May 09, 2019 5:24 am

odroid wrote:
Tue May 07, 2019 12:19 pm
Is "OpenVINO" using the OpenGL-ES API or a generic OpenGL?
I've no idea, this OpenVINO is built by Intel for the Raspberry Pi, so I'd wager it is using what is default on the Pi3. As I said, I've not been a fan of 18.04 anywhere I've tried it.
I see similar regression running my Python code. Since the newest 2019R1 OpenVINO release for Raspbian now supports the C++ interface I thought I'd compare the C++ example to be sure it wasn't a Python issue since the Mate16 and the Pi3 use Python 3.5 and Mate18 uses 3.6. I had to "side load" python 3.5 on the Mate18 Odroid, so I couldn't be sure it wasn't a python issue until now.

User avatar
odroid
Site Admin
Posts: 32677
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 209 times
Been thanked: 364 times
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by odroid » Thu May 09, 2019 9:42 am

User @larrylart seemed to be able to run NCS2 and Coral TPU on the XU4 with openvino. He can give you some idea probably.
viewtopic.php?f=95&t=28177&p=255086#p255086
https://github.com/larrylart/codrive/tr ... ogle_coral

wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Thu May 09, 2019 11:07 am

I'm running them both, NCS & Coral. My issue is the performance on Mate18 is significantly worse than on Mate16.

I've wiped the eMMC of the Mate18 it came with and installed Mate16.

User avatar
odroid
Site Admin
Posts: 32677
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 209 times
Been thanked: 364 times
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by odroid » Thu May 09, 2019 2:50 pm

larrylart wrote:
Tue May 07, 2019 7:05 pm
NCS2 performace on XU4 for mobilenet SSD with openvino was around 19FPS, that using a default c++ demo.
Larry used Mate 18.04 and his test result was 19FPS.
Can you share your test result?
We really want to know how/why 18.04 is much slower than 16.04.

wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Fri May 10, 2019 12:46 am

odroid wrote:
Thu May 09, 2019 2:50 pm
larrylart wrote:
Tue May 07, 2019 7:05 pm
NCS2 performace on XU4 for mobilenet SSD with openvino was around 19FPS, that using a default c++ demo.
Larry used Mate 18.04 and his test result was 19FPS.
Can you share your test result?
We really want to know how/why 18.04 is much slower than 16.04.
I'm a bit confused here, Larry's Github you linked is using the Coral TPU, I plan to try it as soon as I get a chance. But USB cameras will have an influence on the frame rate, I've some that are faster and some slower than the Pi Camera Module when running test codes on the Pi3B, but none of mine are as good as his Logitech, I really only have them for the occasional Skype video conference I'm required to participate in.

My Python code using Mobilenet-SSD https://github.com/wb666greene/AI_enhan ... /README.md sampling 5 Onvif netcams achieves:
## Odroid XU-4, Mate16:
## NCS OpenVINO ~8.5 fps
## 2 NCS OpenVINO ~15.9 fps
## NCS2 OpenVINO ~15.5 fps
## Mixing NCS2 and NCS gets ~8.9 fps barely better than the NCS alone and worse than using 2 NCS. Something weird here.

I forgot to write down the numbers for tests on Mate18 but it was lower by ~1.5 fps if I remember right. This observation was my motivation to try the OpenVINO C++ example code that led to creating this thread. Even on i7-4500U system Mate18 is slower than Mate16 with my Python code using NCS and v1 SDK (doesn't support NCS2) by ~1 fps. The NCS v1 SDK didn't care about Python 3.6 vs 3.5.

I'd like to know why Mate18 is worse as well, although my Python fps decrements aren't quite as dramatic as the OpenVINO C++ sample code. I'll follow up as soon as I've had a chance to run Larry's Coral TPU code on my Mate16 system, although our different Webcams may confound the issue. I'm motivated to do it as I'm just starting up the Coral learning curve.

User avatar
odroid
Site Admin
Posts: 32677
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 209 times
Been thanked: 364 times
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by odroid » Fri May 10, 2019 8:32 am

Thank you for the very detail description. I have to agree near 20% of regression couldn't be acceptable.

Generally speaking, Python 3.6 must be faster than 3.5. https://www.youtube.com/watch?v=JJ3PJ6YUgJ4
But I guess that some portion of the NCS SDK might not be well optimized for Python 3.6 or there are some missing features.

wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Thu May 16, 2019 3:42 am

I'm trying to run Larry's code, but the instructions are less than clear.

doing: ./tools/make/build_rpi_lib.sh

I get this error:
In file included from ./tensorflow/lite/core/api/op_resolver.h:20:0,
from ./tensorflow/lite/core/api/flatbuffer_conversions.h:24,
from tensorflow/lite/core/api/flatbuffer_conversions.cc:16:
./tensorflow/lite/schema/schema_generated.h:21:37: fatal error: flatbuffers/flatbuffers.h: No such file or directory

I've no idea what package flatbuffers.h belongs to :(
but I doubt its the only missing dependency.

I'll do a bit of Googling to see if I get lucky, but at the moment I don't have time to go down a rabbit hole to build someone else's code.


Here is a very easy to use Python3 tutorial example for the Coral USB TPU: https://www.pyimagesearch.com/2019/05/1 ... celerator/

I've run it on Pi3B+ Raspbian, , Odroid XU-4 Mate16.04, and an i3-4025U Mate16.04

results (running via ssh -X):
Pi3B+ Raspbian: ~6.5 fps
XU-4 Mate16: ~19.8 fps
i3 Mate16: ~46.4 fps

I kept a dd copy of my 32GB eEMC, after writing it to an SD card and running this sample code I get ~19.4 fps, but Python performance was not the motivation for starting this thread. OTOH Python is my preferred tools right now for this project so its not worth putting a lot of time into it at the moment.

But I'l lbe happy to try Larry's GitHub code to help evaluate the issue, if I can get it to build without wasting a lot of time. Since I've got Mate18 booted at the moment I try building Larry code again here and report back if I have success.

Edit: Got the exact same build error repeating the steps on Mate18.04 (this image was pre-loaded on the eEMC I ordered along with my first XU-4)

User avatar
odroid
Site Admin
Posts: 32677
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 209 times
Been thanked: 364 times
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by odroid » Thu May 16, 2019 9:23 am

Contact Larry to check whether we can help you or not because we have zero experience of deep-learning area.

wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Fri May 17, 2019 1:51 am

odroid wrote:
Thu May 16, 2019 9:23 am
Contact Larry to check whether we can help you or not because we have zero experience of deep-learning area.
I've posted an issue on his GitHub.

wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Wed May 22, 2019 12:36 pm

With a lot of help from Larry I got his Coral TPU code to compile and run on my Mate16 XU-4. I used his tensorflow library compiled on Mate18 just as I'd used the Raspbian OpenCV libraries for the OpenVINO C++ tests on Mate18 and Mate16.

Mate18 fps results / Mate16 fps results:
NCS 6.56/8.22 = 79.8%
NCS2 8.38/11.5 = 72.7%
Coral 24.9/28.6 = 87.1%

On these C++ sample codes, to me these are significant performance decrements. YMMV.

User avatar
odroid
Site Admin
Posts: 32677
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 209 times
Been thanked: 364 times
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by odroid » Wed May 22, 2019 1:21 pm

Really appreciate sharing the test result.
15~30% of regression must be a quite big concern even though they have same Kernel 4.14.
Does your latest test have no relation to the Python version?

When we tested some heavy database and server benchmarks, 18.04 showed 4~7% of better throughput.
It is really hard to know which part of the Ubuntu 18.04 libraries or toolchains lower the OpenCV performance significantly.
Do you know any generic benchmark tool to measure the OpenCV performance?

wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Thu May 23, 2019 5:52 am

odroid wrote:
Wed May 22, 2019 1:21 pm
Really appreciate sharing the test result.
15~30% of regression must be a quite big concern even though they have same Kernel 4.14.
Does your latest test have no relation to the Python version?
These C++ code samples have no relationship to my Python3 code.

I might look into USB3 drivers if I had the means to do so. From the version evolution of OpenVINO it appears that Intel is looking to support 18.04 but there is no "official" support for doing so. I'd wager this performance decrement might be the reason.

Some opencv C++ performance tests are described here:
https://github.com/opencv/opencv/wiki/QA_in_OpenCV
But I have no experience with them.

As a user I try to avoid compiling OpenCV when ever possible.

User avatar
odroid
Site Admin
Posts: 32677
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 209 times
Been thanked: 364 times
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by odroid » Thu May 23, 2019 9:03 am

It seems to be something beyond my knowledge. :(

Another possible root cause could be a visualization graphics loads.
Does your OpenVINO test application use the Qt5 library?
Or a just a command line text based application?

wbgreen333
Posts: 15
Joined: Fri Apr 19, 2019 4:27 am
languages_spoken: english
ODROIDs: UX-4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by wbgreen333 » Thu May 23, 2019 11:30 am

None of these C++ examples draw graphics until after inference loop is exited.

User avatar
odroid
Site Admin
Posts: 32677
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 209 times
Been thanked: 364 times
Contact:

Re: Mate18 vs. Mate16 regression?

Unread post by odroid » Thu May 23, 2019 12:01 pm

Thank you for the confirmation.

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 1 guest