[SOLVED] J-Link drivers cause "buffer overflow detected" error at Ubuntu 20.04

Godsmack · Mar 12th 2021, 9:09am

Greetings!

Lately I faced critical error after updating J-Link drivers from 6.44d to 6.98a. After update my system cannot run debug with GDB. Process halted after connection to target with "buffer overflow detected" message.

Setup:
Target: KIT_XMC47_RELAX with on-board J-Link based on XMC4200
Platform: Ubuntu 20.04 LTE (VirtualBox)
Driver: 6.98a (and all previous since 6.94)

I checked available drivers - all from 6.44d (which I initially used) till 6.92 are fine. Problem started at 6.94. Also checked with Ubuntu 18.04 - it's ok there. Also working at Win10.

Log:

Display Spoiler

SEGGER - Alex · Mar 12th 2021, 3:40pm

Hi,

We have seen this before but only in combination with loading an RTOS plugin.
As you also found out it is not a generic problem but only seems to happen under specific Ubuntu versions (18.x is fine, 20.x throws the error).
It is a message from outside GDBServer (probably from GLIBC or so) that ends up in the terminal.
When debugging the issue, there was a system call which triggered the error message but it was not obvious why because the buffer that was passed (to store a path) was way larger than the path that was stored.

We will have a look into this again and check if we can reproduce.
Can you provide the VirtualBox image if needed?
We will try to setup it on our own but in case it is not reproducible, the image would help.

BR
Alex

Godsmack · Mar 16th 2021, 10:05am

Good day and thanks for reply!

I am sure that you will be able to re-create issue since I also tried with another freshly installed VM with same OS and drivers. Anyway I'm ready to prepare another VM instance (more compact than my usual ones) for you once again by first request.

Also I tested latest drivers named V6.98b - problem still present. I am attaching screenshots of system error report -maybe they will be useful in tracking issue.

SEGGER - Erik · Mar 16th 2021, 1:08pm

A clean VM would be appreciated as this could speed things up and allows us to check if the our fix works in your environment as well.
Investigation is scheduled for later this / next week.

Godsmack · Mar 17th 2021, 4:04pm

Good day, Erik.

As I promised - below you can find a link to freshly installed, clean, packed and ready VM.

Display Spoiler

Info:
Software: VirtualBox Version 6.1.18 r142142 (Qt5.6.2)
User and pass: main/main
Sample project path: /home/main/Desktop/Sample/
Toolchain path: /home/main/Downloads/gcc-arm-none-eabi-10-2020-q4-major/
IDE: Visual Studio Code (if it's possible to call it IDE)

Note:
You won't be able to compile project since I made scripts only for Debug tests. Also - this project prepared and compiled in DAVE 4.4.2 (Infineon's IDE) and moved inside VM for debug (this made on purpose to make setup faster and also because I can't share exact project and used libs due to reasons).
Anyway - all described in head post problems were recreated and easy to reproduce - right now installed driver version 6.92 and all works like charm but in Downloads folder you can find latest (per now) 6.98c and if you will replace current driver with it - debug became unavailable.

How to use project:
1) Open VS Code (from Favorite panel).
2) Open Sample folder at Desktop in VS Code.
3) Make sure JLink (in my case that's XMC4700 Relax Kit) attached to VM.
4) Hit F5 button - Debug process will start and will wait at the start of main function.
5) Fill free to Run, use Breakpoints, read-out variables etc.

Godsmack · Mar 29th 2021, 10:26am

Good day!

I tested newest version ( named 6.98d ) - all symptoms are the same.

Any new on this topic?

SEGGER - Erik · Mar 30th 2021, 2:43pm

Please excuse the delay in response. It's been some quite busy days...
It's not forgotten and we plan to have a look at this later this week or next week at the latest.

I will keep you posted.

Best regards
Erik

Godsmack · Apr 15th 2021, 9:51am

Greetings, dear SEGGER team!

2 weeks passed by, any news by this topic?

I tested latest release named V7.00 - I can't see same error but debug doesn't start anyway.

Ivan · Apr 20th 2021, 11:07am

I am experiencing the same issue and would also be very grateful if it was solved

SEGGER - Fabian · May 7th 2021, 1:31pm

Hi,
Sorry for the delay in response.
We have not forgotten about this.

Unfortunately, we do not have any resources available to look into this,
but we will do so as soon as possible.
We have some high priority projects with upcoming deadlines we have to get out of the way first.

We are sorry for any inconvenience caused.

Best regards
Fabian

SEGGER - Alex · May 14th 2021, 11:15am

Hi Godsmack et al,

I just got back on this and tried the VM you provided etc.
First of all:
Thanks for the VM. One of the best prepared support cases ever.
Download, start, description how to reproduce etc. all just perfect!

Godsmack wrote:

Anyway - all described in head post problems were recreated and easy to reproduce - right now installed driver version 6.92 and all works like charm but in Downloads folder you can find latest (per now) 6.98c and if you will replace current driver with it - debug became unavailable.

I tried with with the following versions:
V6.92 (pre-installed in the VM)
V6.98c (from your Download folder)
V7.20a (current release)

With all 3 versions I am able to start a debug session with Visual Studio Code and I am able to step through the code, set BPs, ...
I did not see the "buffer overflow" message a single time.
I also made sure that I use exactly the same version of Virtual Box as you do.

This is weird...
I am running Virtual Box on a Windows 10 x86_64 host but the host should not really matter.

Is there anything else I need to do to reproduce the issue?

BR
Alex

SEGGER - Alex · May 14th 2021, 11:40am

I also checked with another VM (Ubuntu 20.04 + VMWare) I have here but starting GDBServer (same command line as you use)
and waiting for the target connection etc. works fine in this VM as well.
I also checked with
V6.98c
V7.20a

BR
Alex

Godsmack · May 17th 2021, 2:43pm

Good day, Alex!

First of all - thanks for finding time to look into this issue!

After reading your messages I decided to repeat your steps as described:

I rolled back VM to state which I provided.
Tested V6.92 which was installed - works.
Installed V6.98c (from your Download folder) - it works too...

At this stage I was pretty discouraged - I am sure that in VM that I provided the issue has been present. So the hunt begins…
In the following description I will operate only with provided deb packages which are located in the Download folder (V6.92 and V6.98c).

What I didn’t mention in the original description - each new installation was made after complete removal of the previous version. How to make it - in Terminal enter following command:

sudo apt-get purge --auto-remove jlink

Take a note at the “auto” prefix - it is extremely important to make sure that we will remove ALL unused dependencies.
After this command feel free to install V6.98c (from Download folder). Running the debug process after this will cause one of two cases - either terminal stuck on “Connecting to target…” or mentioned in the original post “Buffer overflow” (please check attached screenshot).

I noticed that removing different versions causing removing different amount of dependencies:

Installed V6.92 from deb package
Executed sudo apt-get purge --remove jlink (to see which packages will stay but will be unused we do not use auto prefix)
Command output:

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libncurses5 libtinfo5
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
jlink*
0 upgraded, 0 newly installed, 1 to remove and 71 not upgraded.
After this operation, 82,9 MB disk space will be freed.
Do you want to continue? [Y/n]

Let’s repeat with the newer version.

Installed V6.98c from deb package
Executed again sudo apt-get purge --remove jlink
Command output:

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
jlink*
0 upgraded, 0 newly installed, 1 to remove and 71 not upgraded.
After this operation, 85,6 MB disk space will be freed.
Do you want to continue? [Y/n]

In conclusion - if the installation of the new JLink has been performed in a cumulative manner - all systems are working great. In case of pure installation, the issue is still present.

Best Regards

Godsmack · May 18th 2021, 11:30am

Also tested with latest release V720a:

Removed all installed JLink by executing sudo apt-get purge --auto-remove jlink
Download and installed JLink_Linux_V720a_x86_64.deb
Run VSCode and hit F5
"buffer overflow detected" error (check the attached screenshot)

Log from Output tab bellow:

Display Spoiler

SEGGER - Alex · May 22nd 2021, 11:54pm

Hi,

After executing the "purge" with "auto-remove" I can reproduce the issue.
However, I can only reproduce it when starting a session from within VSCode.
If I start JLinkGDBServer stand-alone but with exactly the same command line as VSCode uses, I *never* get a fault:

Source Code

/usr/bin/JLinkGDBServer -if swd -port 50000 -swoport 50001 -telnetport 50002 -device XMC4700-2048

Another weird thing is that this seems to be the toolchain base address used by VSCode:

Source Code

/home/main/Downloads/gcc-arm-none-eabi-10-2020-q4-major/bin/

But if I try to start GDB from thar directory, I get a fault because of missing dependencies (The dependencies that were removed with the purge...):

Source Code

/home/main/Downloads/gcc-arm-none-eabi-10-2020-q4-major/bin/arm-none-eabi-gdb: error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory

So it looks like not only GDBServer is "broken" after the purge...
Wondering how VSCode debugging is supposed to work at all if even GDB cannot be started.
I am not convinced that GDBServer is really the problem here but I am digging...

Additional info:
Issuing "sudo apt install libncurses5" makes GDB to be usable again but at the same time gets rid of the buffer overflow message.
Things are working fine, as soon as this lib is installed.
From /pro/<PID>/maps I do not see that JLinkGDBServerCLExe is loading libncurses, so I do not see any relation between libncurses and J-Link GDB Server.
However, as even GDB stops working if libncurses is not installed, it is hard to say what goes on here.

SEGGER - Alex · May 24th 2021, 11:29pm

OK, found it.
It actually is not really a buffer overflow but glibc seems to detect it as one.

Fact 1) If you remove libncurses5 from your system as part of autoremove, you make your GDB toolchain half-way unusable because arm-none-eabi-gdb depends on it. As this is a portable and non-installed package, the package manager does not detect that libncurses5 is still "needed".

Fact 2) By forcing fact 1, you trigger the following behavior of VSCode:

Start GDBServer
Try to start GDB which should then open the ELF file and connect to GDBServer

Starting of GDB fails because of the missing dependency.
VSCode sends a kill request to GDBServer.

And there is the bug:
GDBServer has a race condition on close so that it can happen that it calls a socket operation with an invalid socket handle.

Fact 3)
While we will fix the race condition in fact 2) you won't be able to debug after "sudo apt purge --autoremove jlink" because GDB cannot run without libncurses5[i].[/i]
So yes, it is a bug but it only fires in a situation where you would be unable to debug anyhow.

Will be fixed in V7.20c

BR
Alex

Godsmack · May 31st 2021, 10:12am

Good day, SEGGER team.

Yes, I can confirm that after clear installation of V698c debug is not available but after installing libncurses5 all functionality fully restored.

So, for the Ivan and future users here some workaround:
1. Fully remove all installed previously JLink and dependencies by executing:
sudo apt-get purge --auto-remove jlink
2. Install required JLink version downloaded from dedicated page (according to my tests affected versions are from 6.98a to 7.20a)
3. Additionally install libncurses5 by executing following command
sudo apt-get install libncurses5

and debugging back in place.

Thank you Alex for deep dive in the issue!

BR

[SOLVED] J-Link drivers cause "buffer overflow detected" error at Ubuntu 20.04

[SOLVED] J-Link drivers cause "buffer overflow detected" error at Ubuntu 20.04

Godsmack wrote:

Source Code

Source Code

Source Code

Share

Tags