Call Search
     

New to Ham Radio?
My Profile

Community
Articles
Forums
News
Reviews
Friends Remembered
Strays
Survey Question

Operating
Contesting
DX Cluster Spots
Propagation

Resources
Calendar
Classifieds
Ham Exams
Ham Links
List Archives
News Articles
Product Reviews
QSL Managers

Site Info
eHam Help (FAQ)
Support the site
The eHam Team
Advertising Info
Vision Statement
About eHam.net

   Home   Help Search  
Pages: Prev 1 [2] 3 4 5 6 Next   Go Down
  Print  
Author Topic: Interesting LoTW Data  (Read 10710 times)
AA6YQ
Member

Posts: 1574


WWW

Ignore
« Reply #15 on: December 16, 2012, 01:21:32 PM »

If eliminating the write performance bottleneck isn't sufficient, we'll know soon enough.

4 to 6 weeks delay to try a potential fix on a system which is already nearly 2 weeks back-logged is a lot of time in today's world.
I suppose being state-of-the-art is no longer a goal for amateur radio, but if there is even a 25% chance the problem is the software, and the software should be tweaked regularly anyway -- does it not make sound sense to optimize it in the interim?

They are not "trying a potential fix", they are eliminating the current bottleneck.

Exactly how would you suggest optimizing an I/O-bound transaction-based system whose storage subsystem is about to be replaced?

     73,

           Dave, AA6YQ
Logged
W6GX
Member

Posts: 2458




Ignore
« Reply #16 on: December 16, 2012, 04:39:23 PM »

If eliminating the write performance bottleneck isn't sufficient, we'll know soon enough.

4 to 6 weeks delay to try a potential fix on a system which is already nearly 2 weeks back-logged is a lot of time in today's world.
I suppose being state-of-the-art is no longer a goal for amateur radio, but if there is even a 25% chance the problem is the software, and the software should be tweaked regularly anyway -- does it not make sound sense to optimize it in the interim?

They are not "trying a potential fix", they are eliminating the current bottleneck.

Exactly how would you suggest optimizing an I/O-bound transaction-based system whose storage subsystem is about to be replaced?

     73,

           Dave, AA6YQ


Since it takes many weeks to get the hardware I assume this stuff is not off the shelf and not cheap.  I hope this much needed investment will make a significant improvement on its performance.  I would love to see a queue time of no more than one hour.

73,
Jonathan W6GX
Logged
K9AIM
Member

Posts: 992




Ignore
« Reply #17 on: December 16, 2012, 05:07:29 PM »


They are not "trying a potential fix", they are eliminating the current bottleneck.

Exactly how would you suggest optimizing an I/O-bound transaction-based system whose storage subsystem is about to be replaced?

     73,

           Dave, AA6YQ


Is it not possible to analyze and upgrade any software bottleneck(s) while waiting for the new hardware to arrive? 
Let me give a crude example which may belie my ignorance:

if a mail delivery system relies on a vehicle that cannot get out of 1st gear, but also involves a sorting process that also limits throughput speed, could not the sorting process be revamped while waiting for the new engine and transmission upgrade to occur?

What chance do you give the hardware upgrade of completely solving the bottleneck(s)?  Also, do you know what kind of server and system LoTW uses?  If so, is there any potential drawback to posting that info here?

Thanks & 73,

Rob K9AIM
Logged
AA6YQ
Member

Posts: 1574


WWW

Ignore
« Reply #18 on: December 16, 2012, 07:20:20 PM »


They are not "trying a potential fix", they are eliminating the current bottleneck.

Exactly how would you suggest optimizing an I/O-bound transaction-based system whose storage subsystem is about to be replaced?


Is it not possible to analyze and upgrade any software bottleneck(s) while waiting for the new hardware to arrive? 

Yes, it is: one could build and validate a performance model, and then use this model to anticipate the system's behavior with the upgraded storage subsystem. Even if knowledgeable resources were available to accomplish this task, building and calibrating such a model from a standing start would be unlikely to produce useful results in a time frame meaningful to this discussion.

Let me give a crude example which may belie my ignorance:

if a mail delivery system relies on a vehicle that cannot get out of 1st gear, but also involves a sorting process that also limits throughput speed, could not the sorting process be revamped while waiting for the new engine and transmission upgrade to occur?

Here's a more relevant automotive analogy: your racing team has decided to replace your car's engine to gain more horsepower; compared to the current engine, the new engine has a different torque curve, a different weight, and a different center of gravity. Can you optimize the car's handling while you're waiting for the new engine to arrive?

What chance do you give the hardware upgrade of completely solving the bottleneck(s)? 

As I have previously posted, the throughput of every transaction-based software system is limited by a performance bottleneck; eliminating this bottleneck always reveals the next bottleneck. If the ARRL's assessment that LotW's throughput is being limited by its SAN-based I/O subsystem is accurate, then the SSD-based upgrade they've ordered should eliminate this bottleneck. Whether the new level of throughput enabled by this upgrade will be sufficient, I do not have the information required to make a competent prediction.

Also, do you know what kind of server and system LoTW uses? 

Please explain how you could make the assertions you've made in this thread without knowing the answers to these basic questions.
Logged
K9AIM
Member

Posts: 992




Ignore
« Reply #19 on: December 16, 2012, 08:11:57 PM »


Here's a more relevant automotive analogy: your racing team has decided to replace your car's engine to gain more horsepower; compared to the current engine, the new engine has a different torque curve, a different weight, and a different center of gravity. Can you optimize the car's handling while you're waiting for the new engine to arrive?

thanks for the more relevant analogy

Also, do you know what kind of server and system LoTW uses? 
Please explain how you could make the assertions you've made in this thread without knowing the answers to these basic questions.

easy.  a little knowledge is a dangerous thing   Grin  that said, the only real assertion i made is that waiting on the hardware as the fix seems to me to be an incomplete approach to the problem.  the recent LoTW explanation of the lost uploads suggested the system was written over a decade ago when none of the present IT team were there and that the ins and outs of how it works has not really been understood until recently (when the root of the lost log issue was uncovered).  given that, a plan to analyze and streamline the present LoTW system on the software end seems in order to a layman like me. Afterall, the amount of users and uploads has had to have grown exponentially since a decade ago and will continue to do so (unless someone else trumps the ARRL here which is not likely since they are the organization most turned to by amateur radio operators who exchange QSL's and seek awards.  Also, i was assuming software technology improvements over the last decade would also suggest the system could benefit in terms of throughput speed through software upgrade(s).  Huh

73,

Rob K9AIM 
Logged
AA6YQ
Member

Posts: 1574


WWW

Ignore
« Reply #20 on: December 16, 2012, 08:32:14 PM »


Here's a more relevant automotive analogy: your racing team has decided to replace your car's engine to gain more horsepower; compared to the current engine, the new engine has a different torque curve, a different weight, and a different center of gravity. Can you optimize the car's handling while you're waiting for the new engine to arrive?

thanks for the more relevant analogy

So can you optimize the car's handling while you're waiting for the new engine to arrive?

Also, do you know what kind of server and system LoTW uses?  
Please explain how you could make the assertions you've made in this thread without knowing the answers to these basic questions.
Easy.  a little knowledge is a dangerous thing   Grin  that said, the only real assertion i made is that waiting on the hardware as the fix seems to me to be an incomplete approach to the problem.  the recent LoTW explanation of the lost uploads suggested the system was written over a decade ago when none of the present IT team were there and that the ins and outs of how it works has not really been understood until recently (when the root of the lost log issue was uncovered).  given that, a plan to analyze and streamline the present LoTW system on the software end seems in order to a layman like me. Afterall, the amount of users and uploads has had to have grown exponentially since a decade ago and will continue to do so (unless someone else trumps the ARRL here which is not likely since they are the organization most turned to by amateur radio operators who exchange QSL's and seek awards.  Also, i was assuming software technology improvements over the last decade would also suggest the system could benefit in terms of throughput speed through software upgrade(s).  Huh

Knowledge of this kind is not dangerous; not understanding the difference between what you know and what you don't know is what's dangerous.
« Last Edit: December 16, 2012, 08:34:49 PM by AA6YQ » Logged
K9AIM
Member

Posts: 992




Ignore
« Reply #21 on: December 16, 2012, 08:46:12 PM »

Here's a more relevant automotive analogy: your racing team has decided to replace your car's engine to gain more horsepower; compared to the current engine, the new engine has a different torque curve, a different weight, and a different center of gravity. Can you optimize the car's handling while you're waiting for the new engine to arrive?
thanks for the more relevant analogy
So can you optimize the car's handling while you're waiting for the new engine to arrive?

not in your analogy where the software is likened to the handling of the automobile (unless you could simulate through modeling how the new engine will change the weight & center of gravity, etc.)

Knowledge is never dangerous; not understanding the difference between what you know and what you don't know is what's dangerous.

right, what we know is infinitesimal compared to that which we know not, and we human animals are bitten with the bug of mistaking belief for knowledge.  that said, contemplation can be a lot of fun.  are you open to the idea the hardware upgrade by itself may not be an adequate solution?  also, are you suggesting that trying to come up with a software upgrade now is premature, or just that we don't have sufficient information about the system and its problem(s)?
Logged
AA6YQ
Member

Posts: 1574


WWW

Ignore
« Reply #22 on: December 16, 2012, 09:09:47 PM »

Knowledge is never dangerous; not understanding the difference between what you know and what you don't know is what's dangerous.
right, what we know is infinitesimal compared to that which we know not, and we human animals are bitten with the bug of mistaking belief for knowledge.

Effective engineers never make that mistake.

that said, contemplation can be a lot of fun. 

Fun for who? For those who must spend time debunking irresponsible claims and speculation, it's not much fun at all.

are you open to the idea the hardware upgrade by itself may not be an adequate solution?  also, are you suggesting that trying to come up with a software upgrade now is premature, or just that we don't have sufficient information about the system and its problem(s)?

I've already stated that while upgrading the storage system should eliminate the current throughput bottleneck, I do not have the information required to predict whether the resulting throughput will be acceptable.
Logged
K9AIM
Member

Posts: 992




Ignore
« Reply #23 on: December 16, 2012, 09:18:18 PM »

are you open to the idea the hardware upgrade by itself may not be an adequate solution?  also, are you suggesting that trying to come up with a software upgrade now is premature, or just that we don't have sufficient information about the system and its problem(s)?

I've already stated that while upgrading the storage system should eliminate the current throughput bottleneck, I do not have the information required to predict whether the resulting throughput will be acceptable.

hence my hope that those who do have access to that information are open to the possibility that the hardware upgrade may not being sufficient to resolve the bottleneck and are already exploring potential software upgrades.  Given the pace of new technology and the trend toward increased usage of LoTW, at some point such an upgrade will be necessary anyway.

73, Rob K9AIM
Logged
AA6YQ
Member

Posts: 1574


WWW

Ignore
« Reply #24 on: December 16, 2012, 09:25:21 PM »

are you open to the idea the hardware upgrade by itself may not be an adequate solution?  also, are you suggesting that trying to come up with a software upgrade now is premature, or just that we don't have sufficient information about the system and its problem(s)?

I've already stated that while upgrading the storage system should eliminate the current throughput bottleneck, I do not have the information required to predict whether the resulting throughput will be acceptable.

hence my hope that those who do have access to that information are open to the possibility that the hardware upgrade may not being sufficient to resolve the bottleneck and are already exploring potential software upgrades.  Given the pace of new technology and the trend toward increased usage of LoTW, at some point such an upgrade will be necessary anyway.


"those"? Do you understand the level of staffing the ARRL has made available for LotW development and maintenance? A fraction of one person.
Logged
K9AIM
Member

Posts: 992




Ignore
« Reply #25 on: December 16, 2012, 09:40:16 PM »

Do you understand the level of staffing the ARRL has made available for LotW development and maintenance? A fraction of one person.

no, I had taken K1ZZ's mention of "the present IT staff" below to mean there was a team available to spend some of their time on that...

Quote from: David Sumner K1ZZ Chief Executive Officer ARRL November 28, 2012
<snip>
When LoTW was designed more than a decade ago -- long before the present IT staff was here -- an assumption was made as to how many logs could possibly be in the queue at a given time. The assumption was based on users uploading their most recent QSOs perhaps once a week or once a month. The environment in which LoTW now operates is quite different from that assumption, in that many users now upload logs with small numbers of QSOs in them, almost in real time. This creates a much larger number of separate logs
When a log is uploaded, it is identified by a file name that is assigned by the user. Because there is no way to avoid duplication of file names that are assigned in this fashion, the LoTW system renames each file. Because of the unusual processing delay -- combined with the dramatic increase in the number of submitted logs -- the system began to run out of unique identifiers for the log files. This resulted in a file sometimes being renamed with an identifier that had already been assigned to a log that was still in the queue, causing the earlier log to be overwritten.

Once the problem was identified, designing a fix was relatively easy. It should be in place by 2359 UTC November 28. Because the number of overwritten logs is relatively small, we have decided to keep the system available for use, even though this may result in a few more logs being lost until the fix is in place. <snip>

 
Logged
AA6YQ
Member

Posts: 1574


WWW

Ignore
« Reply #26 on: December 16, 2012, 09:47:05 PM »

Do you understand the level of staffing the ARRL has made available for LotW development and maintenance? A fraction of one person.

no, I had taken K1ZZ's mention of "the present IT staff" below to mean there was a team available to spend some of their time on that...


Not only is there no "team", there's less than one full-time person assigned to LotW development and maintenance. You jumped to a badly incorrect conclusion.
Logged
K9AIM
Member

Posts: 992




Ignore
« Reply #27 on: December 16, 2012, 09:55:40 PM »


Not only is there no "team", there's less than one full-time person assigned to LotW development and maintenance. You jumped to a badly incorrect conclusion.


i will grant you that a fraction of one person does not a team make.
Logged
N4CR
Member

Posts: 1662




Ignore
« Reply #28 on: December 16, 2012, 10:10:41 PM »

4 to 6 weeks delay to try a potential fix on a system which is already nearly 2 weeks back-logged is a lot of time in today's world.
I suppose being state-of-the-art is no longer a goal for amateur radio, but if there is even a 25% chance the problem is the software, and the software should be tweaked regularly anyway -- does it not make sound sense to optimize it in the interim?

If they have decided to make hardware changes they are probably holding off on software changes. This is typical. If you only change one thing at a time, you see results and you know what caused those results. If you change more than one thing at a time, you see results and now you wonder what it was that made that happen.

I encourage my implementation teams to not make multiple system changes at the same time whenever it is avoidable.

Moving quickly with many changes when a system is in trouble is often counterproductive.
Logged

73 de N4CR, Phil

We are Coulomb of Borg. Resistance is futile. Voltage, on the other hand, has potential.
K9AIM
Member

Posts: 992




Ignore
« Reply #29 on: December 16, 2012, 10:21:34 PM »

so i guess we will know more by the end of January...  
« Last Edit: December 16, 2012, 10:27:25 PM by K9AIM » Logged
Pages: Prev 1 [2] 3 4 5 6 Next   Go Up
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!