eHam.net - Amateur Radio (Ham Radio) Community

Call Search
     

New to Ham Radio?
My Profile

Community
Articles
Forums
News
Reviews
Friends Remembered
Strays
Survey Question

Operating
Contesting
DX Cluster Spots
Propagation

Resources
Calendar
Classifieds
Ham Exams
Ham Links
List Archives
News Articles
Product Reviews
QSL Managers

Site Info
eHam Help (FAQ)
Support the site
The eHam Team
Advertising Info
Vision Statement
About eHam.net



[Articles Home]  [Add Article]  

CWoIP Goes the Butterfly

from Ulrich Steinberg, DJ8GO / N2DE on May 30, 2010
View comments about this article!

CWoIP Goes the Butterfly

In case you have not read my previous articles on the development of an advanced CW memory keyer: The “Butterfly” in the title refers to the name of a microprocessor board which is at the heart of the keyer. When I started this project, more than 5 years ago, little did I anticipate that creative users and I myself would keep coming up with ideas for improvements and new features for such a long time and with no end in sight. Quite a few times I was convinced that everything doable had been done and the keyer surely had reached its final state, and I even expressed that thought in a few articles here – no more ;-)

This project started out as a humble prototype device which was first described on eHam (http://www.eham.net/articles/11749) and which improved in leaps and bounds ( http://www.eham.net/articles/14970 and http://www.eham.net/articles/16608) to eventually become a commercial product, the Begali CW Machine. The microprocessor core of the device has remained the same, making even the first devices that I built fully compatible with the latest incarnation, which has fashionably slimmed down and is just slightly larger than a pack of cigarettes.

Because of that backward compatibility many users have contributed ideas for improved and new functions over the years, which were distributed online to all other users through an online update mechanism. Some of the best ideas, I have to admit, came from users, and I would never have dreamt them up myself.

For some time the CW Machine has had the ability to key a transmitter over an Internet link (http://www.eham.net/articles/19605). This works amazingly well with a remotely controlled rig, and, although the underlying mechanism is quite sophisticated, actually setting it up is straight forward. An extension of this capability, similar at first glance but technically quite distinct, is a new function that was added recently, and which to the best of my knowledge does not exist in any commercial keyer on the market: CW over IP, the “CWoIP” in the title. This allows two CW Machine users to connect their devices over an Internet link and have a QSO off the air. For beginners this is a great way to practice with an elmer without the fear of embarrassment that a real QSO may cause, and for advanced users this is a way to ramp up their speed with a sparring partner without the constraints of a regular QSO. Unlike “real life” you have a perfect display of both sides of the conversation on a computer screen, much more precise than any CW reader could do it. You can hide that display to take a peek later on to see if you got everything right (or if you are too embarrassed to be reminded of your own keying mistakes ;-)

In Dayton this year (2010) it was great fun to puzzle visitors at the Begali booth who tried out Piero’s keys on several CW Machines, and who were wondering how the software could possibly respond sensibly to their QSO fragments – some of them thought that this had to be the smartest simulator on the planet when it was only me on another CW Machine a few feet away, linked through a wireless connection. (in a sense it was a QSO on the 2.4 GHz band used by wireless network adapters ;-)

The data stream that has to flow over the Internet between the two devices is simple, and therefore the keying speed is not restricted in CWoIP mode. The CW Machine will do from 5wpm to 75wpm when used as a “regular” keyer attached to your rig, and it will do the same over an Internet link – without any QRM to break your concentration at higher speeds. Obviously the two parties may have set their keyers to a different speed, and you want the receiver to hear the signal at the speed that it was sent at. Therefore, not only do you have to transmit the actual characters over the Internet but also the speed at which they were created, so that they can be faithfully reproduced by the CW Machine on the receiving end. There also has to be some “heartbeat” signal that lets the two devices know if the link goes down or becomes unbearably slow, and a signal that lets you know that the other side has ended the connection. But that’s about it – the data protocol is quite simple.

Much trickier is the issue of achieving a smooth CW stream at the receiving end even under ideal conditions. This is a consequence of the fact that CW characters vary greatly in length and a character can only be transmitted AFTER it has been formed by the sender. This would result in unnatural gaps in the CW signal, even over a perfect link, as you will see in the following example.

We assume, for simplicity, that the transmission over the Internet takes a constant short time without appreciable fluctuation. Let’s assume we want to transmit the characters E0 (E zero). If you remember the definition of Morse code timing, an E takes 4 time units (one dit and three units for the trailing character space), and the zero takes 22 time units (5 dahs, 4 elementary spaces between the dahs, and 3 units for the trailing character space). It is important to realize that each character is recognized AFTER its trailing character space; at any earlier point in time there could always follow another dit or dah to make it a different character – so, the jury is out on the character, so to speak, until its trailing space has been detected. Back to our E0 example. The upper line represents the timing at the sending end, and the lower line represents what the receiver would hear. The vertical bars represent the action when the character is fully formed and transmitted to the other side:

	E    0
TX:	=---|===-===-===-===-===---| 
RX:	    |=---                  |===-===-===-===-===---

As you can see, even with these idealized conditions of a transmission that takes almost no time (or the same amount of time for each character), there is quite a gap on the receiving end after the E although the sender uses perfect spacing. The receiver reproduces the E, but since it’s such a short character it is done long before the 0 is completed and transmitted by the sender, creating this gap. This happens every time when a character is followed by a longer character. Things get even more complicated in real life because the transmission time may fluctuate. To make sure that the CW stream comes out on the receiving end with proper spacing, some sophisticated adaptive buffering techniques have to be used.

Using the CWoIP function has been fun, and I actually often let my CW Machine run when I am not at home, taking messages from friends, which will be waiting on screen when I come back – a very unique answering machine for CW ops. Of course, you might justifiably say that CWoIP is not ham radio; but it does require CW proficiency which was a signature skill of ham radio operators not too long ago – and you can always connect your laptop computer over a wireless link to your router: there’s the radio ;-) Just kidding: no, it’s not classical ham radio, and it does not require conquering QRM and QRN and QSB on a radio link, but it is a viable way to practice CW with a buddy in preparation for “the real thing” or just for fun.

The development of this device has been an amazing experience, and I have come to realize that the CW Machine users and I will probably never run out of ideas for improvements or new functions, and this may not be the last article here on the CW Machine. Just the other night I was sitting in my backyard, gazing at the stars, my mind wandering, and I thought about the development of the CW Machine from its humble beginnings to the sophistication that it has now, and then a strange thought hit me …

Member Comments:
This article has expired. No more comments may be added.
 
CWoIP Goes the Butterfly  
by K9MHZ on May 30, 2010 Mail this to a friend!
Great, another ad disguised as a discussion.

Buy some ad space in QST.

 
RE: CWoIP Goes the Butterfly  
by KC0FNS on May 30, 2010 Mail this to a friend!


Currently, I don't have a butterfly CW machine (but it sounds like I would like it). But I do have a paddle, a computer and some programming skills.

I enjoyed the explanation, and wondered if the specifications for the CW over IP are open for reproducing and modification? Is there an open source CWoIP app that implements this for inspection and modification? Did you use XML to describe the senders speed, call sign and other information?

Neat idea and it sounds like a lot of fun to use. That is why we do ham radio!

73 OM,
Mark
 
RE: CWoIP Goes the Butterfly  
by K0BG on May 30, 2010 Mail this to a friend!
The real challenges are always in the details!

I never really liked writing software, as there is a lot of tedious repetition. Rather, I enjoyed the challenges of debugging the program as that takes a whole lot more brain power.

Alan, KØBG
www.k0bg.com
 
RE: CWoIP Goes the Butterfly  
by K4KRW on May 30, 2010 Mail this to a friend!
"Rather, I enjoyed the challenges of debugging the program as that takes a whole lot more brain power."

If it takes a lot of brain power to debug a program, it wasn't written very well. Good programs give meaningful appropriate messages to users and technicians and are produced in such a way that they are easy to debug.

I too at times enjoy 'working puzzles'. I've found some doozies in the past. But, I'd rather not have to. A little brain power up front saves a lot of wasted brain power later.

73,

Richard
K4KRW
 
RE: CWoIP Goes the Butterfly  
by KC7NOA on May 30, 2010 Mail this to a friend!
Well ... I bought a K1EL keyer .. but thanks for the AD.
 
RE: CWoIP Goes the Butterfly  
by K0BG on May 30, 2010 Mail this to a friend!
Richard, the point I was trying to make wasn't as clear as I would have liked it.

Anyone can learn to write code, but for some it is a tedious undertaking, especially if you're writing directly in machine code as I used to do. I can't tell you how excited I got, when a decent C compiler for the Macintosh became available.

That left the task at hand, doing the logic, more important than ever. The author pointed out a good example in his E with following Ø problem. I'm sure most folks could figure out what was being said, but writing the code to circumvent the problem is a lot more fun, to me at least.

Technically, the aforementioned problem isn't a bug, it's a graphics problem. Those can sometimes be buggers too (no pun intended).

Alan, KØBG
www.k0bg.com
 
RE: CWoIP Goes the Butterfly  
by N2DE on May 30, 2010 Mail this to a friend!
The problem I highlighted is really an issue of queuing theory: how much do you have to delay the start of the transmission at a minimum to smooth out those gaps, and how do you handle situations where you don't want to delay the start of the transmission? How do you cope with fluctuations in the IP transfer times without unduly increasing the buffer (and the resulting delays)?

The trick is, of course, to write code that doesn't have bugs in the first place ;-) (wish that were possible) A system like the CW Machine is not trivial because much of the functionality is performed by the device and written in very dense RISC Assembler, and complementary functions are performed by code in the Windows program. To write such a multi-platform system is really 50% debugging and 50% coding ...
 
CWoIP Goes the Butterfly  
by W7LRY on May 31, 2010 Mail this to a friend!
Good article and a great machine. I know because I have one and have used it. It is people like Ulrich that have developed Ham Radio equipment and contributed greatly to Ham Radio technology. I notice that some complain the article is an AD. I wonder what they have done to contribute.
 
RE: CWoIP Goes the Butterfly  
by W9OY on May 31, 2010 Mail this to a friend!
Interesting concept

This is definitely the direction Ham radio is going. I could see a dozen uses for a device like this, like repackaging this into a rotor controller that sits on a tower and allows you to WIFI the commands to the rotor/s. One wifi module could easily handle dozens of rotors, and those dozens of rotors could be accessed from all over the world. It totally eliminates that stack of rotor controller boxes sitting on the desk and the miles of control wire needed between the tower and control boxes and reduces the lightning risk

I was just considering yesterday how hard it would be to send CW over the internet for a remotely controlled station and apparently it isn't that hard

If you had a bunch of friends who built a super station on a given band like a 4 square on 160 or 80 with some fancy receiving antennas at one station, and a guy with a high tower and 3 stacked beams on 40 another guy with a stack on 20 etc etc you could form a syndicate and use each others "super" stations remotely. This dramatically reduces the cost but shares the wealth

For example if there were 160 stations syndicated on the west coast and the east coast The west coaster could access the gray line 3 hours earlier and the east coaster could access it up to 3 hours later

Thanks for the hard work you have put into this project

73 W9OY
 
RE: CWoIP Goes the Butterfly  
by W9AC on May 31, 2010 Mail this to a friend!
The best solution to the problem is to form the keyed element, and synchronize it with the companion keyer on the other end of the link. By pacing keyed elements rather than characters the "zero" dilemma is avoided (e.g., the pause after forming the zero in KE0ZZ).

This has been a long-standing issue with the K1EL remote software. But the reality is that the delay of waiting for the zero character is really not that terrible when receiving. I have recorded my own Internet QSOs and when listening back, the delay needed for formation is nearly a "non-issue," but is a slight annoyance.

Another important consideration is that "auto-spacing" should be engaged so that the host keyer can accurately decode the remote keyer. Otherwise, sloppy character sending cannot be interpreted accurately over the link.

These are all manageable issues, but the system that can accurately time keyed elements over a link rather than formed characters will have the competitive advantage.

Paul, W9AC
 
RE: CWoIP Goes the Butterfly  
by N2DE on May 31, 2010 Mail this to a friend!
Sending the contact close/open information over the Internet does not really solve the buffering issue and has several disadvantages. There are many minor fluctuations in the transfer times over a network, and CW would become almost unreadable unless you buffer several dits and dahs. And, as you have pointed out, spacing has to be perfect so that the receiver can properly decode the signal - that is not an issue when you send characters instead of dits and dahs.

I agree that the gaps without buffering, as highlighted in the E0 issue, are a minor annoyance when you copy the text with the decoder between your ears, but I am a perfectionist and want it to look perfect on the screen at the receiving end, too ... ;-)
 
RE: CWoIP Goes the Butterfly  
by W9AC on May 31, 2010 Mail this to a friend!
Not the simple open and closure times -- others are using that technique with mixed results. I am referring to the adaptive timing and synchronization of keyed elements -- not just the open/close time. I have to believe that someone will have a solution for perfectionists like us.

A look-ahead buffer at the host end may be part of the solution as long as it does not introduce unusually long transmission delays.

In addition to using paddles over the Internet, I also use my netbook PC's keyboard, typically when I'm sitting at an airport terminal. Since the ear does not need to time what's being sent, full QSK is possible. The transmission itself is delayed over the Internet, but one can still hear in between keyed elements while typing.

I enjoy using both types of remote CW modes.

Paul, W9AC
 
RE: CWoIP Goes the Butterfly  
by N2DE on May 31, 2010 Mail this to a friend!
The transmission delays with my technique are cut to the bare minimum, just avoiding gaps under normal circumstances, and I have "quick start" logic in there which lets you send even a single character (like, e.g., "?") without having it wait in the buffer until the pipeline is primed. In most situations the transmitter lags by one or two characters. The CW Machine supports both, paddles and your keyboard - but I personally feel much more at home with a paddle for QSOs.

 
RE: CWoIP Goes the Butterfly  
by K9MHZ on May 31, 2010 Mail this to a friend!
>>>by W7LRY on May 31, 2010 I notice that some complain the article is an AD. I wonder what they have done to contribute.<<<<



So who's lamenting what he's developed or "contributed"? Looks like a nice item, good on him. But if he's making a profit (good on him again), take it to QST or CQ and advertise like everyone else. Were you comfortable with the STEPPIR antenna guy describing on here how "convenient, versatile, high quality" his antenna is for his DXpedition?

Contribute away, develop, and innovate....but man-up and buy some real ad space.

 
RE: CWoIP Goes the Butterfly  
by W9AC on May 31, 2010 Mail this to a friend!
> "...and I have "quick start" logic in there which lets you send even a single character (like, e.g., "?") without having it wait in the buffer until the pipeline is primed..."

Excellent technique. I'll need to give that a try one day if I ever depart from using the K1EL remote system.

Paul, W9AC
 
RE: CWoIP Goes the Butterfly  
by N2DE on May 31, 2010 Mail this to a friend!
... and if you are using a keyboard or stored messages there is no delay: since a character is known the moment you press that key or retrieve it from memory, I can send it over the link in parallel to creating it locally. So all these buffering and delay issues come only into play when you use a paddle. (In the CW Trainer, which I am discussing here, I don't support the keyboard, since it is intended as an educational tool to learn CW with a paddle. But the more comprehensive Remote Option for the CW Machine has that capability.)
 
Email Subscription
You are not subscribed to discussions on this article.

Subscribe!
My Subscriptions
Subscriptions Help

Other Recent Articles
Laketown Park in Kenner to Host Coast Guard Auxiliary Radio Day:
Nationwide Amateur Radio Operators Simulated Emergency Testing Oct. 11:
Ham Radio is Still the Old Standby In Emergency Situations:
Amateur Radio Operators to Host Oct. 4 Expo in Georgetown:
Wilkinsburg Ceremony to Commemorate World's First Wireless Broadcast: