Wetware vs. AI Software - Pursuit of Ultimate Morse Decoder

[NI0C]

Mauri,
In another thread, you mentioned you were interested in some comparisons between your software and human decoding.  When you develop some relevant sound files, please share them with us; I'll see what I can do with them (in the spirit of John Henry, Johnny cash's "steel drivin' man").
73,
Chuck  NI0C

Hi Chuck

I posted some sound files with -10 dB to +20 dB SNR @ 2kHz BW.  See my latest blog post on CER vs. SNR testing http://ag1le.blogspot.com/2013/09/new-morse-decoder-part-1.html.  The sound files are here: https://www.dropbox.com/sh/2j9chxsizxjeqgp/Gj97asYzpW.

73
Mauri AG1LE

Mauri,

Many thanks for these sound files.  I just found your posting last night, so have not had much time to listen to them, but my first impression is that there's a huge difference in going from -10dB to -8dB, and to a lesser extent to -6dB.  I'm not good at copying a continuous stream of 50 wpm code groups , but I am able to "grab" some of them and get them down (similar to copying single code signs on Rufz or CWFreak).   I'll try to give you something more quantitative on your blog.

One suggestion might be to record some 20 or 25 wpm code groups, to get more human participants in your experiment.  This might also help you separate out the error rate due to SNR from error rate due to reaching the upper limits of a human subject's code proficiency.

73,
Chuck  NI0C

[AG1LE]

One suggestion might be to record some 20 or 25 wpm code groups, to get more human participants in your experiment.  This might also help you separate out the error rate due to SNR from error rate due to reaching the upper limits of a human subject's code proficiency.

73,
Chuck  NI0C

Chuck
I posted  a set of files  -10db ...20dB SNR @2kHz at 20 WPM speed.  Also, I checked again - the original files had 40 WPM speed. 
The file names are  rand<X>db<Y>wpm.wav where X is  SNR and  Y is WPM, files are here https://www.dropbox.com/sh/2j9chxsizxjeqgp/Gj97asYzpW

Do these work for you?

73
Mauri AG1LE

[NI0C]

Mauri,

Thanks for the new set of files.  I spent some time this morning with the 20 wpm set (20 wpm is about my typing speed!) and was able to do solid copy (essentially zero CER) on the -6 dB and stronger signals. My CER was around 5% at -8 dB, but I fell apart at -10 dB, with only perhaps 50% copy.  This is a very preliminary "quick look," based on copying only a minute or so from each file, (I didn't bother with any of the files stronger than + 6 dB). 

I'm sure there are others with better "ears" than mine who could do better, and perhaps at lower signal to noise ratios.  Your -10 dB files at 20 and 40 wpm are good practice files for me, and might help me train my ears. 

73,
Chuck  NI0C   

[AG1LE]

I spent some time this morning with the 20 wpm set (20 wpm is about my typing speed!) and was able to do solid copy (essentially zero CER) on the -6 dB and stronger signals. My CER was around 5% at -8 dB, but I fell apart at -10 dB, with only perhaps 50% copy.  This is a very preliminary "quick look," based on copying only a minute or so from each file, (I didn't bother with any of the files stronger than + 6 dB). 

Thanks Chuck!
 
I will use these data points as reference. Human auditory system seems to have a pretty good filtering capability what comes to Morse code type signals. To achieve similar results with this software I would need to get roughly 16 dB improvement by filtering the original signal that has 2KHz bandwidth. Since the noise in these files is white noise 16 dB translates to roughly 50 Hz bandwidth [16 dB = 10*log10(2000/50)].

I did some work back in January trying to compare the impact of matched filtering vs. FFT filtering in FLDIGI. The plot below shows the big improvement that filtering brings on CER figures:


Once I get this new algorithm integrated better with FLDIGI I can re-run the above CER vs. SNR tests to see how much improvement the Bayesian probabilistic correlator-estimator provides. According to the doctoral thesis I used as the source for this algorithm there should be ~ 4 ... 6 dB improvement.

Thanks again for taking the time for this experiment!

73
Mauri AG1LE

[NI0C]

Mauri,
I wish you continued success with your project.  Although I've taken a couple of graduate courses in digital signal processing, that was over 30 years ago, and I'm pretty rusty.  Nevertheless, it was fun helping you out in a small way using manual CW copying skills that I have tried to maintain.
73,
Chuck  NI0C

[W5LZ]

The human brain/ear can make an enormous amount of 'adaptions' that no 'AI' can do (at present).  That 'wet' computer can associate what it hears with what should have been heard where a set of algorithms can't unless it's a humongus number of algorithms.  I wish you good luck with your project.
 - Paul

[GILGSN]

I don't have an Arduino board but the software is posted in Github: https://github.com/ag1le/morse-wip.

Thanks Mauri, I will keep an eye on it. Very interesting in any case. I use a Bayesian filter for spam deletion and it works great, so... Your program being in C it should be adaptable to many platforms.. An arduino has 32K or program memory.. I'me just starting messing with it..

Gil.