Call Search
     

New to Ham Radio?
My Profile

Community
Articles
Forums
News
Reviews
Friends Remembered
Strays
Survey Question

Operating
Contesting
DX Cluster Spots
Propagation

Resources
Calendar
Classifieds
Ham Exams
Ham Links
List Archives
News Articles
Product Reviews
QSL Managers

Site Info
eHam Help (FAQ)
Support the site
The eHam Team
Advertising Info
Vision Statement
About eHam.net

   Home   Help Search  
Pages: Prev 1 [2]   Go Down
  Print  
Author Topic: Wetware vs. AI Software - Pursuit of Ultimate Morse Decoder  (Read 43270 times)
NI0C
Member

Posts: 2437




Ignore
« Reply #15 on: September 28, 2013, 09:15:51 AM »

Mauri,
In another thread, you mentioned you were interested in some comparisons between your software and human decoding.  When you develop some relevant sound files, please share them with us; I'll see what I can do with them (in the spirit of John Henry, Johnny cash's "steel drivin' man").
73,
Chuck  NI0C

Hi Chuck

I posted some sound files with -10 dB to +20 dB SNR @ 2kHz BW.  See my latest blog post on CER vs. SNR testing http://ag1le.blogspot.com/2013/09/new-morse-decoder-part-1.html.  The sound files are here: https://www.dropbox.com/sh/2j9chxsizxjeqgp/Gj97asYzpW.

73
Mauri AG1LE

Mauri,

Many thanks for these sound files.  I just found your posting last night, so have not had much time to listen to them, but my first impression is that there's a huge difference in going from -10dB to -8dB, and to a lesser extent to -6dB.  I'm not good at copying a continuous stream of 50 wpm code groups , but I am able to "grab" some of them and get them down (similar to copying single code signs on Rufz or CWFreak).   I'll try to give you something more quantitative on your blog.

One suggestion might be to record some 20 or 25 wpm code groups, to get more human participants in your experiment.  This might also help you separate out the error rate due to SNR from error rate due to reaching the upper limits of a human subject's code proficiency.

73,
Chuck  NI0C

Logged
AG1LE
Member

Posts: 139


WWW

Ignore
« Reply #16 on: September 28, 2013, 06:27:51 PM »


One suggestion might be to record some 20 or 25 wpm code groups, to get more human participants in your experiment.  This might also help you separate out the error rate due to SNR from error rate due to reaching the upper limits of a human subject's code proficiency.

73,
Chuck  NI0C

Chuck
I posted  a set of files  -10db ...20dB SNR @2kHz at 20 WPM speed.  Also, I checked again - the original files had 40 WPM speed. 
The file names are  rand<X>db<Y>wpm.wav where X is  SNR and  Y is WPM, files are here https://www.dropbox.com/sh/2j9chxsizxjeqgp/Gj97asYzpW

Do these work for you?

73
Mauri AG1LE
Logged
NI0C
Member

Posts: 2437




Ignore
« Reply #17 on: September 29, 2013, 10:19:37 AM »

Mauri,

Thanks for the new set of files.  I spent some time this morning with the 20 wpm set (20 wpm is about my typing speed!) and was able to do solid copy (essentially zero CER) on the -6 dB and stronger signals. My CER was around 5% at -8 dB, but I fell apart at -10 dB, with only perhaps 50% copy.  This is a very preliminary "quick look," based on copying only a minute or so from each file, (I didn't bother with any of the files stronger than + 6 dB). 

I'm sure there are others with better "ears" than mine who could do better, and perhaps at lower signal to noise ratios.  Your -10 dB files at 20 and 40 wpm are good practice files for me, and might help me train my ears. 

73,
Chuck  NI0C   
Logged
AG1LE
Member

Posts: 139


WWW

Ignore
« Reply #18 on: September 29, 2013, 11:17:17 AM »

I spent some time this morning with the 20 wpm set (20 wpm is about my typing speed!) and was able to do solid copy (essentially zero CER) on the -6 dB and stronger signals. My CER was around 5% at -8 dB, but I fell apart at -10 dB, with only perhaps 50% copy.  This is a very preliminary "quick look," based on copying only a minute or so from each file, (I didn't bother with any of the files stronger than + 6 dB). 
Thanks Chuck!
 
I will use these data points as reference. Human auditory system seems to have a pretty good filtering capability what comes to Morse code type signals. To achieve similar results with this software I would need to get roughly 16 dB improvement by filtering the original signal that has 2KHz bandwidth. Since the noise in these files is white noise 16 dB translates to roughly 50 Hz bandwidth [16 dB = 10*log10(2000/50)].

I did some work back in January trying to compare the impact of matched filtering vs. FFT filtering in FLDIGI. The plot below shows the big improvement that filtering brings on CER figures:


Once I get this new algorithm integrated better with FLDIGI I can re-run the above CER vs. SNR tests to see how much improvement the Bayesian probabilistic correlator-estimator provides. According to the doctoral thesis I used as the source for this algorithm there should be ~ 4 ... 6 dB improvement.

Thanks again for taking the time for this experiment!

73
Mauri AG1LE
Logged
NI0C
Member

Posts: 2437




Ignore
« Reply #19 on: September 30, 2013, 06:05:32 AM »

Mauri,
I wish you continued success with your project.  Although I've taken a couple of graduate courses in digital signal processing, that was over 30 years ago, and I'm pretty rusty.  Nevertheless, it was fun helping you out in a small way using manual CW copying skills that I have tried to maintain.
73,
Chuck  NI0C
Logged
W5LZ
Member

Posts: 477




Ignore
« Reply #20 on: September 30, 2013, 04:01:52 PM »

The human brain/ear can make an enormous amount of 'adaptions' that no 'AI' can do (at present).  That 'wet' computer can associate what it hears with what should have been heard where a set of algorithms can't unless it's a humongus number of algorithms.  I wish you good luck with your project.
 - Paul
Logged
GILGSN
Member

Posts: 208




Ignore
« Reply #21 on: September 30, 2013, 10:59:41 PM »

Quote
I don't have an Arduino board but the software is posted in Github: https://github.com/ag1le/morse-wip.

Thanks Mauri, I will keep an eye on it. Very interesting in any case. I use a Bayesian filter for spam deletion and it works great, so... Your program being in C it should be adaptable to many platforms.. An arduino has 32K or program memory.. I'me just starting messing with it..

Gil.
Logged
Pages: Prev 1 [2]   Go Up
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!