[time-nuts] Long life products, obsolete components, and code 4 parts. RE: HP Cesium Standards in the International Atomic Time Scale, the legend of Felix Lazarus, and the "top cover

[paul swed]

hello to the group.
I really am enjoying the stories and in reality history about a great
company that I am very proud to actually own a lot of its technology.
Though purchased for $/lbs at hamfests and such. All needing TLC and all of
it teaching me more then a few lessons on how to do things.
So though the stories center around real world trade offs. The fact is the
guts were still very good. Says the guy with home brew crystal references.
Chuckle.

My one and only story was circa 1994-98 and video. Was at HP to see demos
of their video server. Always thought it would be a great place to work.
But going through the sea of cubes and seeing what it would actually be
like settled that forever.
Keep up the stories please.
Regards
Paul
WB8TSL

On Mon, Dec 31, 2018 at 12:37 AM Rice, Hugh (IPH Writing Systems) <
hugh.rice at hp.com> wrote:

> I’ll add a bit to Rick’s story, from my manufacturing engineering
> perspective.
> I was hired into PFS manufacturing engineering in 1984, specifically to
> work with Roberto (Robert) Montesi on the 5061B product.   Roberto was the
> production engineer on the 5061A, and acting “project manager” of our
> little two man development team.  We were funded by manufacturing, as Rick
> noted, but sat in the R&D lab for about 18 months as we redesigned
> (updated) a bunch of stuff on the 5061A.
>
> As I mentioned a few weeks ago, Roberto was a very good engineer (compared
> to me at least), and a great mentor.  One story he told me about himself
> that I recall:  He was originally from Nicaragua, and somehow wound up in
> the US Army during Vietnam, spending some time in combat there as a GI.
>  At one point the Army wanted to send him to officers candidate school,
> since he scored so well on all the tests.   He was a smart guy, with
> perfect English.   Well into the process, they finally realized that he
> wasn’t a US citizen, and thus couldn’t be an officer.    He seemed to make
> it through the whole Vietnam experience with minimal PTSD (as far as I
> could tell), and would tell an interesting (and likely cynical) war story
> now an then.   Like Rick said, Roberto kept his head down, and we sat in a
> shared work area and did our 5061B thing, surrounded by the team working on
> the new 5350,51,52 microwave counters, led by their very energetic project
> manager Bob Renner.   The real R&D guys treated us well, even though we
> were 2nd class production guys.   (Not too many years before this, R&D
> engineers and production engineers were not on the same pay scale, and
> really were second class in HP eyes.   The feeling of not being a “full”
> engineer still lingered in 1984.)
>
> As Rick said, PFS products like the cesium standards were cash cow
> products, and didn’t have a R&D staff at all.   All the “upgrades” were
> funded by manufacturing, to keep this product line viable.    The whole
> development effort was about extending the production life of the 5061A.
>  We were selling about 15/month, with an average price of about $35K.  The
> gross margins were very high (sales price – material costs), and the
> product line delivered 4 or 5 million in gross profits to the division a
> year.    It was well worth having a couple of manufacturing engineers
> freshen things up to keep this cow healthy.    And Roberto was still the
> production engineer for the 5061A during this time, so kinda doing double
> duty.
>
> My job on the 5061B was to redesign the clock display and the battery
> charger.  These were both part of the popular time-keeping option 003,
> which was primarily the 1pps output circuit.  The battery backup was to
> prevent the 1PPS signal from losing syc. If there was a power
> interruption.   I recall the battery charger had a huge mica capacitor that
> couldn’t be purchased any more, and a crazy design with obscure TTL
> counters.   The clock display was even crazier.  Not nixie tubes,  but two
> or three circular PC boards driving LED displays, and again obscure ICs
> that were hard to procure.   Hard to build, and really expensive.   (More
> on these in another story on another day.)
>
> Roberto redesigned the frequency divider module (5MHz in; 10MHz, 1MHz,
> 100kHz out – another odd design rooted in 5060 history), the A3 power
> regulator board, and some stuff internal to one (both?) of the high voltage
> power supplies, used for the Ion Pump and Electron Multiplier.   Maybe some
> other things too.  For all of these, obsolete components was the driving
> force.
>
> By 1984 standards, there were some really crazy circuits still in the
> instrument (still another story for another day), but as Rick said, in low
> volume manufacturing, if it isn’t broken, don’t fix it.  In the case of the
> 5061, don’t even think about touching it.
>
> Rick’s memory of the management dynamics are similar to mine.   A 5061A to
> 5061B “upgrade”, particularly if funded by manufacturing, was easy to get
> approved.    Entire new developments were hard to justify.    The division
> was under a lot of financial stress in the 1980s.  Peace was breaking out
> as the cold war was winding down, and DOD spending, which drove a lot of
> instrumentation sales, was shrinking.     Digital oscilloscopes and
> synthesized frequency generators were obsoleting the need for frequency
> counters, the majority of the divisions revenue.    PFS was profitable, but
> zero growth.   We also build laser interferometers, which did amazing high
> precision displacement measurements, but they weren’t growing either.
>  While profitable, the division revenue was shrinking maybe 10% per year.
>   In the 8 years I was there, headcount went from about 1500 to 500
> people.   Management was desperate to fund new products that would lead to
> growth.  I recall the general manager at the time (Jim Horner), having a
> metric for every new development on how much growth it would contribute to
> the division.  It was never enough.   Redesigning the 5061A yielded zero
> growth (the demand for cesium standards was pretty flat) and thus not a
> priority.    A very light touch by manufacturing to keep it viable was
> appropriate.
>
> This email chain has unleashed a flood of memories from 30 years ago.
>  Hopefully a few of you find this walk down memory lane interesting.     I
> have a few more stories in the que if any of you are still interested.
>
> Hugh Rice
>
>
> From: time-nuts <time-nuts-bounces at lists.febo.com> On Behalf Of Richard
> (Rick) Karlquist
> Sent: Sunday, December 30, 2018 7:35 AM
> To: Discussion of precise time and frequency measurement <
> time-nuts at lists.febo.com>; Magnus Danielson <magnus at rubidium.dyndns.org>
> Cc: magnus at rubidium.se
> Subject: Re: [time-nuts] Long life products, obsolete components, and code
> 4 parts. RE: HP Cesium Standards in the International Atomic Time Scale,
> the legend of Felix Lazarus, and the "top cover
>
> The HP way (AFAIK) was as follows:
>
> They were making the 5061A and the
> default philosophy was don't fix it
> if it ain't broke. However, products
> reach a tipping point. In the case
> of the 5061A, the obsolescence of the
> Nixie tube was the straw that broke
> the camel's back. But there were a
> bunch of other issues that had also
> accumulated a critical mass.
>
> I was hired into HP in 1979 to work in the
> Precision Frequency Sources R&D section
> to work on the 10816 rubidium. That
> project was eventually cancelled by a "new sheriff
> in town" event upstairs, and took the
> section with it. So they had to somehow
> boot leg the 61B without an R&D section.
>
> A production engineer
> named Robert (I forgot his last name) was
> the project manager. He basically tried
> to keep his head down and not attract a
> lot of attention. I am thinking that all
> the money came out of the production engineering
> budget.
>
> Another HP way thing is that we would
> go from A to B in order to get the clock
> running on the end of support life. Upper
> management would be not be suspicious of an
> A to B, as opposed to a new product number,
> which would be a red flag. The cesium line was
> to be run as a cash cow, period. Len pulled a
> rabbit out of the hat when he got permission for
> the 5071A.
>
> So the 61B was a bridge product to keep the
> plane flying until the 71A came out. It
> basically contained no gratuitous improvements,
> only stuff that had to be fixed.
>
> Rick
>
> On 12/30/2018 5:23 AM, Magnus Danielson wrote:
> > Dear Hugh,
> >
> > Many thanks for another nice post from the good old times.
> > Was a nice morning reading.
> >
> > I didn't know that the 5061B was rebuilt with removing odd parts in
> > mind, but it makes sense. Interesting system with Code 1 to Code 4.
> >
> > Cheers,
> > Magnus
> >
> > On 12/29/18 5:36 AM, Rice, Hugh (IPH Writing Systems) wrote:
> >> My “Test and Measurement” days with HP, from 1984 to 1992, were all in
> manufacturing (a.k.a. production) engineering. A major task was dealing
> with the endless list of obsoleted components, since many of our products
> had designs dating back 10 or 20 years, into the wild west early days of
> semiconductors and integrated circuits.
> >>
> >> In addition to Frequency and Time products (which we called “PFS” –
> Precision Frequency Sources), HP’s Santa Clara Division (SCD) also had the
> frequency counter product line. I managed the production engineering team
> for counters from 1988 to 1992; the job that I had to pass the R&D
> engineering new hire interview to qualify for. This technology was invented
> in the 1950s and even with many new models and upgrades, we still were
> shipping “classics” products from the early 1970s in low volume in about
> 1990. The 5340 microwave counter and 5328 universal counters come to mind.
> We kept raising the prices, because we had newer, better, cheaper counters
> for sale. But the old ones kept selling because they were designed into
> some DOD test system, and the hassle of designing in a new instrument was
> more expensive than buying an new (but obsolete) counter for our customers.
> The parade of obsolete components seemed to never end on these old units. I
> recall talking to the marketing manager, Murli Thurmali (sp?) about
> obsoleting some of these products, and he would wisely respond: “Tell me
> how you are going to replace the million dollars of lost revenue.” The
> manufacturing manager, Chuck Taubman, would likewise say: “Our margins are
> well over 50% on these products, that money pays overhead, which is our
> salaries. Show me $500K in cost savings before we obsolete them.” Turns out
> that even though they were a hassle, it was relatively easy money, so we
> kept building and selling them.
> >>
> >> The PFS products were similar in this regard. The product line had
> largely been developed in the 1960s and 1970s, volumes were low, but prices
> and margins were high. Yeah, they took some effort to keep in production,
> but the development was done, and it was good money. HP was a business
> after all, and if we didn’t make money, we didn’t have jobs. The was a
> great education for me, brand new to management, learning that HP may be a
> cool technology company, but we only had jobs as long as the business was
> profitable, and preferably growing. Nothing was guaranteed.
> >>
> >> HP instituted a system of “Codes” for parts, to measure how well we
> were designing our products for long production lives and low materials
> management overhead costs. Code 1 was best. Industry standard parts
> available from many sources cheaply. Code 2 were OK to use. Code 3 was
> something really special, and needed a good reason to include. Code 4
> brought the scorn of procurement engineers, and brought significant
> management review.
> >>
> >> The easy way out for production engineering to deal with obsoleted
> component was a life time buy. The Materials group hated this, because they
> had hundreds of other parts already on life time buys. What if they get
> lost or damaged, or the last batch was defective, or the product lasted
> longer than we expected? A product like the 5061A, at ~200 build per year,
> was a typical challenge. 10 more years of life? Buy 2400 parts? Perhaps
> double it to 5000 parts. The response from component buyers was easy to
> predict: “But VendorX wants $2.31 for this ancient transistor. We’re not
> tying up $10K in one part. We have dozens of parts like this, we can’t
> afford all this inventory.” So we would try harder. Maybe a 2N222A, or a
> 2N3904 will work. Procurements loves these parts. We’d try them out, and
> hope we didn’t miss something in the qualification. New parts never had the
> same specs at the old parts, and the original designer was long gone, and
> design intent documentation non-existent. I bet half the time the old
> transistor just happened to be on the engineers bench back in 1969, worked
> fine, and he just used it. The Code 1,2,3,4 process was designed to
> discourage this kind of design thinking.
> >>
> >> When we upgraded the 5061A Cesium Standard to the 5061B in 1984-85, the
> primary objective was to eliminate all the code 3 and code 4 parts.
> Designing out all the old stuff wound up being a fantastic education in
> component technologies, reading and interpreting data sheets, dealing with
> vendors, worrying about inventory control and so on. Our attitude was
> trying to make a product we could ship indefinitely, even though it was
> already over 20 years old. We had a history of selling PFS instruments for
> decades, and we were preparing for decades more.
> >>
> >> Bob kb8tq wrote: “In the case of the 5071, I’d bet a pretty good brand
> of six pack that nobody on the planet would have guessed 20 years ago that
> it still would be in production today.”
> >>
> >> Well, I can’t prove that Bob would lose this bet (Maybe Rick K could),
> and I didn’t work on the 5071. But for PFS products, in production
> engineering, we had been building and selling these instruments for
> decades, with no end in sight. Volumes were low, so they didn’t get
> redesigned very often. I’ll bet the same six pack that the 5071 team felt
> it would be a VERY long time before HP designed a replacement for the 5071.
> >>
> >> Rick – any memories you can share?
> >>
> >> Happy New Year,
> >>
> >> Hugh Rice
> >>
> >>
> >> From: time-nuts <time-nuts-bounces at lists.febo.com<mailto:
> time-nuts-bounces at lists.febo.com>> On Behalf Of Bob kb8tq
> >> Sent: Monday, December 24, 2018 9:35 AM
> >> To: Discussion of precise time and frequency measurement <
> time-nuts at lists.febo.com<mailto:time-nuts at lists.febo.com>>
> >> Subject: Re: [time-nuts] HP Cesium Standards in the International
> Atomic Time Scale, the legend of Felix Lazarus, and the "top cover
> >>
> >> Hi
> >>
> >> Indeed back at Motorola, a lot of that stuff got transferred into the
> engineering stock room
> >> after a while. Just how that worked out budget wise …. one wonders ….
> >>
> >> Bob
> >>
> >>> On Dec 24, 2018, at 11:53 AM, jimlux <jimlux at earthlink.net<mailto:
> jimlux at earthlink.net<mailto:jimlux at earthlink.net%
> 3cmailto:jimlux at earthlink.net>>> wrote:
> >>>
> >>> On 12/24/18 5:36 AM, Bob kb8tq wrote:
> >>>> Hi
> >>>> The gotcha is - if you have a very unique part in a device and it
> goes away, how
> >>>> many years of stock do you buy on the “last chance” order?
> >>>> In the case of the 5071, I’d bet a pretty good brand of six pack that
> nobody on the
> >>>> planet would have guessed 20 years ago that it still would be in
> production today.
> >>>
> >>> EOL buys for a product line are plausible. But if you're building
> one-off (or limited quantity)- maybe not. At work (JPL) there's a whole
> aspect to sparing that's kind of subtle - you get funded per mission, and
> it has a cost cap at the proposal stage.
> >>>
> >>> Buying extra parts "just because" cuts into your budget - what do you
> give up because you bought extra parts, maybe some engineering hours? or
> test time? - it's easy to say "oh what's a few parts here and there", but
> pretty soon, it's getting to be a big part of your budget.
> >>>
> >>> So you buy enough parts to build what you're going to launch, plus
> enough maybe for an EM or breadboard, and then a few spares in case there's
> some assembly errors, or you need to scrap a board. If the problem happens
> early enough, you've got time to burn some reserves and order more.
> >>>
> >>> The other problem in the space business is that there is a lot of
> desire to re-use known good designs. That part may have been a long way
> from EOL when it was first used, but now, 5-10 years later, maybe it's EOL,
> and there's no obvious "drop in" replacement. Do you redesign, or do you
> buy the last remaining stock and hope for the best?
> >>>
> >>> This tends to be a cascading issue - mission A designs and uses part
> X, and has spares. Smaller Mission B uses the spares to build their widget
> using the Mission A design. They buy a few spares too. Smaller Mission C
> does the same thing. Now we're 10 years in, in some cases still using spare
> parts bought by original Mission A.
> >>>
> >>> I am still using spare connectors and such from Cassini (launched in
> 1997) in things like breadboards at work.
> >>>
> >>>
> >>>
> >>>>> On Dec 24, 2018, at 1:59 AM, Hal Murray <hmurray at megapathdsl.net
> <mailto:hmurray at megapathdsl.net<mailto:hmurray at megapathdsl.net%
> 3cmailto:hmurray at megapathdsl.net>>> wrote:
> >>>>>
> >>>>>
> >>>>> jimlux at earthlink.net<mailto:jimlux at earthlink.net<mailto:
> jimlux at earthlink.net%3cmailto:jimlux at earthlink.net>> said:
> >>>>>> and the "market lifetime" of parts today is much shorter. There are
> lots of
> >>>>>> parts from Hittite that were essentially "run on this line only",
> and when
> >>>>>> they moved geometries, they're never to be seen again.
> >>>>>
> >>>>> Most vendors make a lot of noise before they pull the plug on a
> part. The
> >>>>> usual deal is that they fill all orders placed by a specified date -
> lifetime
> >>>>> buy. Distributors typically send a note to anybody who has purchased
> them, or
> >>>>> maybe only purchased significant quantities.
> >>>>>
> >>>>> If a part isn't expensive, you can afford to buy extras beyond what
> you expect
> >>>>> to need to cover some what-ifs. That probably doesn't cover
> something like
> >>>>> the 5071 being in production for 30 years. But it could give you a
> few years
> >>>>> warning - maybe enough time to find a substitute and/or redesign
> that section.
> >>>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> time-nuts mailing list -- time-nuts at lists.febo.com<mailto:
> time-nuts at lists.febo.com<mailto:time-nuts at lists.febo.com%
> 3cmailto:time-nuts at lists.febo.com>>
> >>> To unsubscribe, go to
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com<
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com><
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com<
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com>>
> >>> and follow the instructions there.
> >>
> >>
> >> _______________________________________________
> >> time-nuts mailing list -- time-nuts at lists.febo.com<mailto:
> time-nuts at lists.febo.com<mailto:time-nuts at lists.febo.com%
> 3cmailto:time-nuts at lists.febo.com>>
> >> To unsubscribe, go to
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com<
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com><
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com<
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com>>
> >> and follow the instructions there.
> >> _______________________________________________
> >> time-nuts mailing list -- time-nuts at lists.febo.com<mailto:
> time-nuts at lists.febo.com>
> >> To unsubscribe, go to
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com<
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com>
> >> and follow the instructions there.
> >>
> > _______________________________________________
> > time-nuts mailing list -- time-nuts at lists.febo.com<mailto:
> time-nuts at lists.febo.com>
> > To unsubscribe, go to
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com<
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com>
> > and follow the instructions there.
> >
> >
> _______________________________________________
> time-nuts mailing list -- time-nuts at lists.febo.com<mailto:
> time-nuts at lists.febo.com>
> To unsubscribe, go to
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com<
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com>
> and follow the instructions there.
> _______________________________________________
> time-nuts mailing list -- time-nuts at lists.febo.com
> To unsubscribe, go to
> http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
> and follow the instructions there.
>