Pages: 1 [2] 3   Go Down

Author Topic: IS LOTW DOWN?  (Read 920 times)

N4UFO

  • Member
  • Posts: 1171
Re: IS LOTW DOWN?
« Reply #15 on: January 02, 2023, 02:52:14 AM »


Many FT8 users upload after each QSO.  This practice is totally unnecessary and the overhead of processing thousands of one QSO files is enormous. A daily upload, or maybe, every 10 to 25 QSOs, would be better and would greatly reduce processing overhead.

I have often wondered if that wasn't causing some problems... and while I might selfishly appreciate 'instant QSLs', I try to do mine at the 'end of a session' and limit my uploads to, at most, a few times per day.
Logged

KT4QD

  • Member
  • Posts: 10
Re: IS LOTW DOWN?
« Reply #16 on: January 02, 2023, 05:49:25 AM »

What cost are you referring to?

There are several ham web interfaces I have worked with recently.  Many of these suggest  that the big cost is connecting at all and until the volume of data gets really insane, a small amount and a big amount aren't much different in cost.  Still, a huge volume of requests is a huge volume of requests.
Logged

WA9AFM

  • Member
  • Posts: 978
Re: IS LOTW DOWN?
« Reply #17 on: January 02, 2023, 06:26:30 AM »

Queue status as of 1425Z is 10.5 hours.
Logged

N2TU

  • Member
  • Posts: 51
Re: IS LOTW DOWN?
« Reply #18 on: January 02, 2023, 10:12:45 AM »

Internal Server Error....LOTW appears down....1812UTC
Logged

EI2GLB

  • Member
  • Posts: 1252
Re: IS LOTW DOWN?
« Reply #19 on: January 02, 2023, 01:20:11 PM »

Think someone is going to have to pop in and either turn it off and on again, or put a few quarters in the meter,

Queue is over 12hrs now,
Logged

WO7R

  • Member
  • Posts: 6042
Re: IS LOTW DOWN?
« Reply #20 on: January 03, 2023, 05:40:32 AM »

Quote
What cost are you referring to?

The cost, in time, of setting up a connection between two computers in the internet. 

The internet is a marvel of technology.  It routinely connects two completely arbitrary computers somewhere in the world and does so despite the fact that a connection between any two computers can easily require a path between the two to be discovered and set up that includes easily thirty or more other computers that have to cooperate.

It even does this given that connecting is a bit of a living thing.  Computers come and go all the time.  The name arrl.org might target a different computer ten minutes from now than it does now.  It mostly doesn't, but the fact that it might means a lot of stuff has to be redone every time two computers connect -- that is, when they create a path to send data between them.  Even if the two computers turn out to be the same physical machines as before, the path between them may be different next time.

This is actually remarkably fast, but if you arrange to do LOTW after every QSO, you are tempting fate.  If I have 100 QSOs, there is also a cost of sheer transmission time to send those 100 QSOs regardless of what the code organizes to do.  That isn't free in terms of time either.  But, if I remake the connection between the machines 100 times to send one QSO each time instead of one connection to send 100 QSOs, the overhead of those  extra 99 connections adds up to something substantial, especially if thousands of my buddies do the same thing.

Conversely, if we organize to "hold back" some number of QSOs and not send every QSO one at a time, we all see less overhead.
« Last Edit: January 03, 2023, 05:42:35 AM by WO7R »
Logged

KJ4Z

  • Member
  • Posts: 1628
Re: IS LOTW DOWN?
« Reply #21 on: January 03, 2023, 04:17:37 PM »

I think what we are witnessing is a predictable outcome of cost optimization and engineering trade-offs.

Normally, LOTW is fine.  At the moment it is behaving like I would have expected a pre-cloud/early-aughts app to behave under bursty loads, a level of performance totally unacceptable in a modern consumer app.  But this isn't that... it's an app designed to serve a niche technical market at (I assume) the lowest practical price.  Being responsive to unexpected swings in demand is quite easy these days but it does require money.  I am guessing this is just one or two servers at ARRL HQ.  Someone has decided it's not worth the extra cost and effort for once or twice a year events.

All that said, I would have been embarrassed to propose to my boss that we address load issues by asking our users to moderate their behavior.   :)
Logged

N4UFO

  • Member
  • Posts: 1171
Re: IS LOTW DOWN?
« Reply #22 on: January 03, 2023, 04:46:18 PM »

Someone has decided it's not worth the extra cost and effort for once or twice a year events.

Clearly you weren't around before they upgraded the servers some years back. You could upload a file and it might take close to 24 HOURS to be processed. This is NOTHING compared to what it was before they overhauled everything and replaced the servers. If it needs it, I am sure ARRL will keep it running. LotW has become an important part of the amateur radio activity.  :)

73, Kevin N4UFO
Logged

KJ4Z

  • Member
  • Posts: 1628
Re: IS LOTW DOWN?
« Reply #23 on: January 03, 2023, 04:59:27 PM »

Clearly you weren't around before they upgraded the servers some years back. You could upload a file and it might take close to 24 HOURS to be processed.

I was around for that, actually, and remember it.  My point was that, after those improvements, the normal state of LOTW is acceptable.  Obviously in those days a pain threshold was crossed and they responded.  We are not there anymore.
Logged

K4HB

  • Member
  • Posts: 1522
Re: IS LOTW DOWN?
« Reply #24 on: January 03, 2023, 05:01:04 PM »

Believe it or not, some people only log once a year. Yep. They play all year, they don't give a rats hat about confirmations, AND just to do us all a favor they dump their logs at the end of the year. And this is what happens.

I believe you're right. Just had a couple of 6M grids worked in June to come in. The queue status was 14 hours this morning, and now down to less than 9. That's progress, and hope there's some more I need in the pile.
Logged

 Click 4 pic
K4HB dot com

WO7R

  • Member
  • Posts: 6042
Re: IS LOTW DOWN?
« Reply #25 on: January 03, 2023, 10:37:58 PM »

LOTW has some serious problems.  I'm not suggesting it is in danger of going out of business.  But it is not obviously a very good business.

1.  Like a lot of web services, all the payment is back-ended.  They offer a free service until you cash in.  There's a lot of stuff on the internet that works like that.  A close analog would be in-app purchases on an iPhone for an app that is free until and unless you decide on some purchases.  Maybe you pay to get rid of ads, for instance.  Or, maybe you pay to get a cool "skin" for your iPhone game character.  Yes, people do that.

2. The economics for LOTW, then, is to get enough people to "convert" from unpaid to paid.  In our parlance, to actually apply for an award.  The "conversion problem" actually a studied problem for people who do things like invent iPhone apps that are free-with-something-to-buy-later.  However, most people do seem to want DXCC and similar awards, so this is the least of their problems, fortunately, particularly with the tie to the CQ WAZ stuff and even the WPX stuff.

3.  One problem that maybe sets LOTW apart a little bit from, say, some iPhone gaming app is that it has some dis-economies of scale.  A substantial fraction of uploaded QSOs, particularly those that go more than 120 days without a match never get one.  Those are still potential money-makers, but the majority never do.  Worse, there are a lot of matches that never get turned in.  I got my 20m SSB QSO with Japan confirmed in the paper days.  Most newly minted DXers get theirs in LOTW about the first week they take up DXing.  The thousands of other matches to JAs (e.g. in contests) are all dead bits whether they match or not.  And, since those dead bits reside in some sort of curated data base the cost per QSO (dead or alive) is much greater than the raw cost of storage.  Maintenance of data bases costs real, skilled people to manage.  It isn't just shoving the next 2 Terabyte Drive into some slot.

4.  The pile of dead bits grows every day.  So do the moneymakers.  But as someone's career in DXing matures, pretty much everything is dead bits.  Since I don't contest much I "only" work a couple thousand Qs a year.  About half confirm in LOTW.  But, of those, since I am north of 2400 on the DXCC Challenge, maybe one hundred at most end up generating LOTW any money.  So, the longer I DX, the lousier a customer I am.  If I contested regularly, I'd be even a worse customer for them at this point.  They actually need a fair amount of turnover in DXing, or at least new blood, to make money.  We old farts are somewhere between OK to lousy to money losers for them.

Separately, I've been told by a couple of sources who are well placed that a lot of LOTW's infrastructure is in C++.  You can't take that to the bank -- I don't really know.  But, based on my sources, I believe it to be true.

C++ remains one of the most venerable and popular languages out there.  If you have those skills, you can get a job, believe me.  I've made good money writing C++ code over the years.  Still, it is also one of the most complex and arcane languages out there.  It would not be my first choice for a language that is nimble and capable of rapid deployment of new function.  But nobody wants to get rid of working code and pay to re-write it. Re-writes happen, but that is deadweight expense no matter how rich the company.  Usually the old code is left around and the new code is added on top.  I worked with FORTRAN (believe it or not) around 2010 because nobody in my company wanted to kill this core of very old, but critical code.  However, I don't think ARRL is doing anything other than in C++ today.  Not sure why (or even if it is true).  If they do stick with C++ (and they could see it as a net advantage somehow), I would predict slower deployment of new function than we would otherwise see.
« Last Edit: January 03, 2023, 10:45:47 PM by WO7R »
Logged

N4UFO

  • Member
  • Posts: 1171
Re: IS LOTW DOWN?
« Reply #26 on: January 04, 2023, 02:26:14 AM »

"particularly with the tie to the CQ WAZ stuff and even the WPX stuff."

Keep in mind some organizations are now accepting PDF 'printouts' of LotW QSL records for their awards. I know AMSAT does it, I've heard WPX is willing to do it. Bottom line, no money stream for ARRL that way.


"I worked with FORTRAN (believe it or not)"

My FAVORITE language!!! In Comp Sci class I once wrote a database access program (the assignment) because I could get the subroutines to access individual bytes the way I wanted. My professor wrote on my printout, "A- does all required, missing some documentation - YOU are a MASOCHIST!"  ;D

Seriously, I was majoring in Comp Sci in the mid 80s and the guys coming up behind me were being taught all the new 'fancy languages' and I realized I would be outdated by the time I graduated. So I took a minor in Comp Sci and swapped over to the easiest degree I could get outta there with. Which at the time was Social Work, but they changed that degree on me over to 'Human Services' and I could no longer get my LSW license. So know I have a BA in Sociology instead... that and a buck will get you a soda out of the cheap machine. - I learned on mainframes and had a job one summer installing PC gear for businesses that ran on CP/M. And it was not simple... you could not just go BUY a serial cable, etc. Had to go to Radio Shack and buy DB25 connectors and wire and solder them up. People have no idea today.


Back to LotW... while your analysis may be correct, I think you are overlooking that ARRL does not just see LotW as a profit business model, but rather as a promotion to ham radio in general and creating an importance to their presence in that promotion. In short, they do LotW to 'up their brand', not to make money. That said, I have no need for the 'paper awards'... they stay in the cardboard tubes up on the closet shelf. But I do like to 'turn in' for my awards and trust me, a 9BWAS can be hundreds of dollars. I look at it as a donation to ARRL. I don't look at it as a business arrangement as that might kick in the 'cheap ham' DNA. But seriously, I appreciate the 'tick marks' and the 'standings' (VUCC) and encourage other 'cheap hams' to see it the same way... a donation. The awards themselves only have as much value as you place in them. I guess I value mine... the award itself, not the paper. So I pay.

KJ4Z, I was being a bit sarcastic there... with your call, one would assume you were around then.  ;) I was more saying for the benefit of anyone who was not, "boy, if you think this was bad... when I was a kid... in the freezing rain and both directions"  :D

73, Kevin N4UFO
« Last Edit: January 04, 2023, 02:39:21 AM by N4UFO »
Logged

KJ4Z

  • Member
  • Posts: 1628
Re: IS LOTW DOWN?
« Reply #27 on: January 04, 2023, 09:28:24 AM »

KJ4Z, I was being a bit sarcastic there

I thought you might be, but I also wanted to make sure I hadn't left the impression that I was criticizing LOTW for what is, in my opinion, a legitimate decision to save running costs.

I think WO7R's key insight is that nothing ever gets deleted from the LOTW database, therefore its operating cost effectively increases year over year.  I would be interested in knowing how many matches "convert."  I have about 10,000 entries on there from 30 years of hamming, and about 60% of those have a match.  I think I have converted about 150 of those to actual credits.  That's probably about $15 in fees?  That probably does cover the actual costs of providing me the service, barely, but it sure doesn't seem like a cash cow.

BTW, a conversion rate of 1%-2% is considered pretty good in a freemium model.
Logged

N4UFO

  • Member
  • Posts: 1171
Re: IS LOTW DOWN?
« Reply #28 on: January 05, 2023, 09:23:36 AM »

FYI, looks like it has finally recovered...

2023-01-05 13:59:02   53   6,733   2,898,302   2023-01-05  13:58:02  (0d 00h 01m 00s ago)
Logged

EI2GLB

  • Member
  • Posts: 1252
Re: IS LOTW DOWN?
« Reply #29 on: January 05, 2023, 11:08:38 AM »

You spoke too soon it's on the way up again, I think there is more than the end of year uploads at play here,

There was a guy who used to post here that was somewhat involved with LoTW (or the upgrade) he was a 6 had his own logging SW is he still around, Sorry can't remember anymore than that,

Trevor
EI2GLB


FYI, looks like it has finally recovered...

2023-01-05 13:59:02   53   6,733   2,898,302   2023-01-05  13:58:02  (0d 00h 01m 00s ago)
Logged
Pages: 1 [2] 3   Go Up