Call Search
     

New to Ham Radio?
My Profile

Community
Articles
Forums
News
Reviews
Friends Remembered
Strays
Survey Question

Operating
Contesting
DX Cluster Spots
Propagation

Resources
Calendar
Classifieds
Ham Exams
Ham Links
List Archives
News Articles
Product Reviews
QSL Managers

Site Info
eHam Help (FAQ)
Support the site
The eHam Team
Advertising Info
Vision Statement
About eHam.net

   Home   Help Search  
Pages: [1]   Go Down
  Print  
Author Topic: FLDIGI Morse decoder SNR to CER testing results  (Read 2785 times)
AG1LE
Member

Posts: 136


WWW

Ignore
« on: January 01, 2013, 10:03:36 PM »

I spent some time today running some tests to find out the limits of FLDIGI v3.21.64  CW decoder in terms of CER (character error rate)  vs.  SNR  (signal-to-noise ratio).

Test results are available here: http://ag1le.blogspot.com/2013/01/morse-decoder-snr-vs-cer-testing.html

I am looking for advice:

1)  Has anybody done similar tests comparing multiple CW decoder software packages with noisy signals?

2)  What other tests or metrics would you recommend to drive further improvements in the FLDIGI CW decoder? 


Happy New Year and 73
Mauri  AG1LE
Logged
N4UM
Member

Posts: 472




Ignore
« Reply #1 on: January 05, 2013, 05:21:23 PM »

I don 't mean to be critical but I think that getting CW decoders to decode poorly sent CW is probably a more important problem to tackle than getting them to decode well sent CW under noisy conditions.  I've used Fldigi's CW decoder on several occasions and it fails to do an adequate job on all but near-perfect code. 
Logged
AG1LE
Member

Posts: 136


WWW

Ignore
« Reply #2 on: January 05, 2013, 06:35:18 PM »

I don 't mean to be critical but I think that getting CW decoders to decode poorly sent CW is probably a more important problem to tackle than getting them to decode well sent CW under noisy conditions.  I've used Fldigi's CW decoder on several occasions and it fails to do an adequate job on all but near-perfect code.  

How to handle noisy conditions is one of the  issues that I would like to solve.   Did you look at this CER/SNR study I did few days ago? http://ag1le.blogspot.com/2013/01/morse-decoder-snr-vs-cer-testing.html.   You can see clearly how important filter bandwidth is to CER (character error rate).   I did some work in  May / June 2012  to implement a prototype Matched filter for FLDIGI  what Dave W1HKJ  re-wrote using faster FFT/FIR based methods.  He also added capability to adjust FFT filter band with with automatic speed tracking.   I should probably run some earlier version of FLDIGI as comparison how much these features have improved the decode.

However,  I would agree with you that decoding poorly sent CW is another very important topic.  Different non-standard rhytms and timing will get FLDIGI decoder confused.  I worked last summer also to implement a SOM decoder feature that is based on self organizing maps neural network algorithm.  As this is still heavily relying on the existing FLDIGI finite state machine that is the heart of the decoder  the improvement that SOM brings is only 5 - 10% in CER depending on the SNR.  

Bayesian framework is a totally new approach to this problem, we use probabilities instead of hard thresholds & limits  and pass those probabilities up in the detection chain.  I am still learning as I go here but this looks like a promising area of research and experimentation.  CW Skimmer is using Bayesian decoder and it does pretty good job in  pile-ups and contest situations.

Any suggestions how I could improve the FLDIGI decoder further?   What kind of metric should be used as the standard to compare against?

73
Mauri AG1LE

« Last Edit: January 05, 2013, 06:44:26 PM by AG1LE » Logged
N4UM
Member

Posts: 472




Ignore
« Reply #3 on: January 06, 2013, 04:50:39 PM »

Quote from AG1LE; "Any suggestions how I could improve the FLDIGI decoder further?   What kind of metric should be used as the standard to compare against?"

I really have no idea how to go about improving the FLDIGI filter ! 

One suggestion regarding a possible standard... (somewhat tongue in cheek)... Get Marti Lane (OH2BH) to copy some known text under controlled sets of adverse conditions... fading, noise, static crashes and QRM near the same frequency.  Then use his performance as a standard for comparison..  He's one of the best and most experienced CW OP's in the world. 

I'm not sure what you mean by a Bayesian framework.  Do you mean using something like a matrix of confusion probabilites based upon how often an "A" is incorrectly decoded as an "N" or a "Q" etc.  Do you mean using a conditional probablility matrix of sequences of letters in standard English etc. I guess it all depends on the type of text you're trying to decode. 
Logged
AG1LE
Member

Posts: 136


WWW

Ignore
« Reply #4 on: January 07, 2013, 02:10:20 PM »

Quote from AG1LE; "Any suggestions how I could improve the FLDIGI decoder further?   What kind of metric should be used as the standard to compare against?"

I really have no idea how to go about improving the FLDIGI filter ! 

One suggestion regarding a possible standard... (somewhat tongue in cheek)... Get Marti Lane (OH2BH) to copy some known text under controlled sets of adverse conditions... fading, noise, static crashes and QRM near the same frequency.  Then use his performance as a standard for comparison..  He's one of the best and most experienced CW OP's in the world. 

I'm not sure what you mean by a Bayesian framework.  Do you mean using something like a matrix of confusion probabilites based upon how often an "A" is incorrectly decoded as an "N" or a "Q" etc.  Do you mean using a conditional probablility matrix of sequences of letters in standard English etc. I guess it all depends on the type of text you're trying to decode. 

What a great idea!  I know Martti from many years (I was living in Finland until end of 1997 and was active ham back then).  Martti  would be fantastic  "gold standard"  - I will send him a note if he would be willing to help here.  Just need to design the control parameters of the system - something like http://www.dxatlas.com/PileupRunner/ might be good? 

On Bayesian framework - see this http://ag1le.blogspot.com/2013/01/towards-bayesian-morse-decoder.html.  I have been working on FLDIGI CW decoder prototype using above ideas.  Currently testing the proto with Dave W1HKJ and few others.

73
Mauri AG1LE
Logged
Pages: [1]   Go Up
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!