[time-nuts] Re: St Veran gravity red-shift misson
Bob Camp
kb8tq at n1k.org
Mon Jul 24 13:19:13 UTC 2023
Hi
Wow, that’s quite a story. Thanks for sharing !!!!
While this is a bit off topic in terms of timing:
Victron does sell “central control†computers to tie their gear together. Some are in the
< $200 range. They will let you manage an ensemble of devices effectively. It also is
worthwhile to tie in something like a BMV-712 shunt based monitor. That will let you
evaluate the battery(s) state accurately.
There are folks who make “hats†for the RPi that do backup power as well as DC/DC
conversion. They claim they are pretty robust. No idea if they would be robust enough
for this sort of thing.
Lithium batteries, while still expensive, are getting less so every day. They are way lighter
than lead acid if you are going to lug them here and there. The Victron gear is just as
happy running them as running various forms of lead acid. They also will happily sell
you setups for 24V if you decide that works better than 12V.
As you might guess, there’s a lot of Victron gear in my RV’s …..
Bob
> On Jul 24, 2023, at 4:06 AM, Magnus Danielson via time-nuts <time-nuts at lists.febo.com> wrote:
>
> Hi all,
>
> So, now this morning we rose early, I was late at 6:30. Quick cleanup of room, breakfast, take antenna down, reconnect small pin-wheel antenna for climb down. That part went well.
>
> Driving down the steep road is harder than the climb. During winther only skis work, and it takes 4 hours. It can be 2-3 m snow. Snow is good, melted snow becomes water for the station, which is used during summer. We saw marmottes and mountain deers. Unnecessarily cute. Fashinating talking to the care-talker, we really hit off well and love his stories. He lives locally and I now kniw his family.
>
> Clock transfer from Observatoire car to my car went eventless... not. We lost power again, now for a new reason. I keep learning things, and the failures is important.
>
> So, I had changed the loading power from 4A to 12A at masimum on the AC charges while at station. Well, there it is safe. In the car, not so much, so the 12 VDC to 230 VAC (125 W) ducked and that caused temporarily outage. I fairly quickly retained power and set charger to reasonable values and things where fine. However, it required me to restart the Raspberry Pi, since I had not had the time and energy to make all the logging work. OK, so hook up screen to RPi for it to boot properly. Massive problems, and it turns out that the screen is dying. I managed to get the environmental logging running, cesium logging, pressure-sensor, and power setup logging. Then pack things up and start the decend from Saint Veran village.
>
> The first stretch, I let Bruno drive, as he has much more experience in mountain driving than me.
>
> Part of the travel down, there was a peep. Turns out that everything was fine, except that the 12 VDC to 230 VAC converter ducked under the heat. So I turned it of and it works. From here the PV panels was far than enough, so no worries. This happened a few times more during the travel, but without major effect on resull.
>
> We stopped for lunch, drove further, and from Grenoble I took over driving. Filling the tank, getting some soft drinks and such, a drive of 800 km to just outside Mons in Belgium, where we where able to have late dinner and charge the batteries. Then drive to the Observatoir Royal de Belgique in Brussles.
>
> Once at place at the Observatory, we had to plan how to get the clock into the laboratory, again while avoiding to power the clock down. Through inventive use of extention cables and using a pair of batteries as power backup and manage the switch of power out-let, we where able to take it out of the car, roll it down the ramp and then through the corridors into the time-lab and eventually slid it in just under the Cesium 1 of the ORB cesium rack.
>
> The safe transfer over to laboratory location and power was successful and at 03:00 we could leave the site.
>
> While the adventure contains several events, the combinations of measures alows us to compensate for iit to some degree. The big and small failures is to learn from, and sharing the gory details of experience is part of the learning experience. Having just 3 weeks on the spare time to build a setup like this is not enough. It remains the main source for the failure.
>
> The general concept of power, GNSS and logging works. Especially power is challenging and being able to alternate between solar panels, AC and car generator is a contributor to the relative success. The lack of automation allowed too much of human error in handling. Much of that ended up being handled without majjor failure of the mission, but it did mean loosing valuable measurement and scientific strength. The loss of measurements on low level is a problem, and we then need to recover that as being measurement after the travel, just to establish the rate of the clock at sea level. Assuming the clock maintains that rate in it's own frame of reference, which we then change as we go up and then go down the mountain, and that we reduce the graviational potential and thus gravitational acceleration. However to show this we need to know the rate of the clock at sea level (ORB lab is at about 100 m) to illustrate the difference in rate. We choose to measure this. Speaking of measuring, we choose to use GNSS common view measurements using the PolaRx4TX as main measurement, and the Mosaic-T/X5 as auxillary. The later would fail to do PNT solutions but continue to measure everything else. Using the 10 MHz as primary signal, and PPS more as auxillary, loss of PPS phase was less of a failure if only the clock state was recovered quickly.
>
> Overall I consider the power-strategy as such as a success. The implementation leaves room for improvement, but considering it was whipped together in three weeks while working full time, the failures and near misses mainly comes from stress and lack of time to integrate things. I was recommended to use the Victron MPPT and AC-chargers, as they have known good quality and also an open and open-source oriented view on things. I had no previous experience of these myself. The MPPT solar panel charger is accessable over both VE Direct electrical interface and over BlueTooth. While I usually is not overly ethusiastic for BlueTooth, this time I was extremely happy to have the ability to monitor over BlueTooh. Using a VE Direct USB adaptor cable, I was able to integrate the monitoring to the InfluxDB and Grafana infrastructure relatively well. That however only happend once up on the Observatory. I should be able to integrate over the BlueTooth, but I did not have the time and energy to do that, but it will be the next thing to attempt, when I reach my summer house.
>
> The thing which really worked well with the power system was the dynamic capabilities of it. The MPPT charger has three power connections, the PV solar panel, the batteries and then the load. Towards the solar panel, it optimize the load by adjusting the current drawn, to find the optimum V/I operational spot to achieve the maxium output power P for each condition. When panels have too little energy, it turns it of. According to Victron recommendations, an AC charger can be added by hooking it in directly on the batteries. What is missing is a better integration between these. While the MPPT was doing great, and the AC charger was doing great while operation each on it's own, the cut-over between them and more importantly handling of the different sources of AC power (house or car) with different charger conditions allowed for human error, as will be illustrated. On both the battery side and the load side I used a RigRunner 4008H power-pole panel. I used power-pole for all 24 V power connections. This allowed me to use unusused cables as backup, it also allowed me to hook up a pair of batteries to directly drive the cesium when needed either as UPS or as DC source when switching AC connector.
>
> Power failures did happen, and here is a list of the cesium failures:
>
> 1. Lab setup
>
> When testing the DC feed to the cesium, pulling the AC failed.
>
> Cause: Forgot to connect the DC cables all the way.
>
> Root cause: Human error, forgot to check and verify the DC level.
>
> Comment: The cesium does not have an indicator of the power status of the AC and DC. It does the power-switching well, but it would help to know.
>
> 2. Transfer from lab to car.
>
> When transfering the setup to the car, in itself a dress rehearsal and test of the concept of moving it over, the cesium failed.
>
> Cause: Connecting the DC cable to the distribution channel was done to a connector with a fuse of 1 A and the cesium pulls about 2 A. Blown fuse expected. Fuse and blown-fuse indication worked.
>
> Root cause: Stress contributed to this, this was done just as I was about to drive 600 km to the summer house.
>
> Recommended correction: Having planned and marked the intended connection point on the panel to the cesium would have avoided the issue.
>
> 3. Short circuit in Grenoble
>
> Driving down from summerhouse to outside Mons was a 1100 km drive. That was successful. Then driving down to Grenoble was in itself also successful. However, lack of sleep was a contributor. When in Grenoble issues had been detected and a failure of the setup experienced. Part of the setup removed to secure the experiment.
>
> Cause: A cable had come loose, so a +24V cable made connections it should not have done. Fuses blown etc.
>
> Root cause: The haste by which everything was assembled prooved that everything was not connected well enough, strapped down enough for the isolation to work. Re-connecting it to the small DC/DC adaptor (for +5V), with more ZIP-ties to secure cables and then isolation and location worked.
>
> Recommended correction: Have more time to spend on securing cables and better isolation would have resolved this.
>
> 4. Drop-out in Grenoble village
>
> Having charged the batteries at the station, and then transfer the setup to the car to have it under charge during the night was done to ensure successful transfer. During transfer between the cars, operating on my car's 12V using the 12V to 230 VAC adaptor, the clock can be transfered without batteries. However, this is done by driving the AC adaptor using the generated AC. This has been a good strategy for all transfers. This drives the 24V bus and everything else depends on it. However, this time it failed.
>
> Cause: The AC charger had been put in 12A charging mode and high voltage level while as the observatory, since on observatory power (in itself a large PV array and battery bank) the load on the 125W DC to AC convertor made it duck, and with that, the 24V system when batteries was disconnected. Reverting to battery shortend the outage. Reducing setting on AC avoided the next power outage.
>
> Root cause: Human error and integration error. Having the AC charger act both as supply for AC and car DC source (though DC/AC converter) made integration harder. There was a good separate car DC outlet, but no suitable 12V to 24V switch supply available, that would have avoided the integration problem with 12V load and 24V load. Integration could have avoided more of the human error and steered things better.
>
> Recommended fixes:
>
> 4.1 Check-lists could have reduced the risk.
>
> 4.2 Better setup would have reduced the risk.
>
> 4.3 Better integration would have reduced the risk.
>
> 4.4 Using the AC supplies ability to operate as power supply rather than charger would have changed the outcome.
>
> 4.5 Keeping charging/supply also from observatory car would have reduced the problem.
>
> 4.6 More time to integrate and test
>
> While this is not an exhaustive set of problems, it illustrates well how lack of time and stress enabled human error and deficiencies in the setup to cause failures. However, considering the many transfer of power, clock transfers, dynamic handling of setups, carrying etc. the general concept itself seems to work really well. It just needs to work out it's teething problems.
>
> I chose to be brutally honest, because then there is more to write and more to learn. I chose to focus on the power in this post, but logging had it's set of challenges. Also, cause of problem was that the screen used with the raspberry pi is about to die, which was realized only on decent in Saint Veran village.
>
> I intend to report more as I have arrived at my summerhouse. I think the logging would be good to explain and do a post mortem on.
>
> It's only by making an honest attempt that one can test how teories work in practice. It is only by honestly report and do full post-mortem that the failures can be converted into useful experience and enable en even smoother operating setup on next run.
>
> Some of what I've build, is really what should go into my basement time-lab too. The difference is really not that large.
>
> I will close the laptop, brush my teeth and pack up to leave the smallest hotel I've ever stayed at.
>
> Cheers,
> Magnus
>
> On 2023-07-23 00:57, James Littlefield wrote:
>> Enjoy your feast. It has been fascinating reading your chronicles of the experiment.
>>
>> On Sat, Jul 22, 2023 at 6:54 PM Magnus Danielson via time-nuts <time-nuts at lists.febo.com> wrote:
>>
>> Hi all,
>>
>> Our time here starts to run out. This is bitter-sweet, because it
>> means
>> leaving a really nice place with a boatload of new friends, good
>> times
>> and for me a whole amount of astronomy I did not expect to see.
>>
>> After doing the last show for the turists, we have moved the clock
>> and
>> batteries over to the observatory car where it will sit over night.
>> Power from the house and maintain the choke ring on the top. All to
>> minimize the things we need to do tomorrow morning, since we need to
>> raise early and then travel down to the village, and then transfer
>> the
>> clock back to my car, and then drive across France and into
>> Brussels, to
>> move the clock into the Observatoir Royal de Belgique lab.
>>
>> It's been very eventless here, but things where fixed in the
>> beginning.
>> So, you would expect things to be fine... well, just as we loaded the
>> clock and locked the car, we saw that one tire is flat, so right now
>> work to change the tire. Luckily we have a spare tire. So, the usual
>> mess ongoing. Had to unload some, but not too much.
>>
>> I have tried to use a TICC, logged a lot of data in parallel to
>> the GNSS
>> common view, and it turns out that the least square fit is for
>> ridicolous litte frequency shift with about 3E-16, so that is
>> probably
>> just the same clock looped back. Ah well. The integration works. What
>> does not work is signal on ChB so I need to investigate that as I
>> come home.
>>
>> We have learned immensly by attempting this project, and just
>> finishing
>> it we try to use the knowledge.
>>
>> I'll setup a webpage on my server to describe as much as I can.
>>
>> The hobby-astronomers have various other background, such as one
>> being a
>> math professor, so I showed him my fast least square work, just to
>> see
>> what he thought. Kind words was exchanged. Other than that, lots of
>> physics, math etc. have been discussed.
>>
>> Car tire changed, things reloaded. Everything is fine.
>>
>> Now we prepare for the final night feast.
>>
>> Cheers,
>> Magnus
>>
>> On 2023-07-19 03:06, Christopher Hoover wrote:
>> > Huzzah! Yes, please send updates.
>> >
>> >
>> > On Sun, Jul 16, 2023 at 1:41 AM Magnus Danielson
>> <magnus at rubidium.se>
>> > wrote:
>> >
>> > Dear Christoffer,
>> >
>> > Many thanks. Yesterday we maanged to drive from Grenoble up into
>> > St Veran village, the highest village in Europe. This had me
>> > driving mountain roads like nothing I've ever done before, so it
>> > was a very intense experience. I even drove through Italy,
>> so the
>> > second new country for me to drive in.
>> >
>> > From St Veran village my car was not suitable, as the final
>> climb
>> > is on a road which is not very good, so we had to use the
>> > Observatory jeep and transfer the clock over to the shock
>> > absorber. This is quite an elaborate dance, but thanks to the
>> > modularity of the setup where where able to safely transfer
>> it up.
>> > We used a small pin-wheel antenna to record the remaining rise.
>> >
>> > At the observatory, I made sure that batteries was recharged
>> > before we transitioned furteher. It was quite an elaborate dance
>> > to move it up into the lab. We also installed a choke ring
>> antenna
>> > on the roof of the observatory.
>> >
>> > I use my Mosaic-T and ORB brought their PolaRx4TR. This provide
>> > redundancy, which is good, since we discovered that the
>> cable from
>> > the antenna-splitter over to the PolaRx4TR can have a loss
>> of contact.
>> >
>> > Having spare cables and adapters have helped, so we had to be a
>> > bit innovative to extend the cable, but I weather secured the
>> > extension setup, so we should be fine.
>> >
>> > We maintained the cesium operational from Grenoble. I will
>> > investigate when we lost state, but it was before Grenoble,
>> so we
>> > should have good data from that, even if Cesium date and time is
>> > off, we do not measure that so I can adjust that later today.
>> >
>> > I now go into a clean-up, so start additional loggings I did not
>> > managed to start manually because of stress. I aim to make those
>> > autostart, it's not hard, but one had to prioritize hard.
>> Power to
>> > the cesium is key, GNSS recording operational second, then
>> logging
>> > of cesium, environment, PV & batteries further down the
>> line. The
>> > key to mission have been achieved. I can clean up the other
>> stuff
>> > while here. I want that additional environmental and system
>> > logging to work for the rest of the mission.
>> >
>> > So, we have not failed the mission yet. It's not been
>> perfect, but
>> > with some clean-up action and experience built, we can improve
>> > robustness.
>> >
>> > The Observatoir de Saintt Veran is originally built for
>> > Observatoir de Paris, and they used to have two domes in Paris,
>> > but the light poultion makes it fairly useless. So, they built
>> > this observatory and moved one of their domes up here. Yesterday
>> > late, they where walking to that dome saying "Want to see
>> Saturn?"
>> > and I just tagged along. It was not the best of days, since
>> it was
>> > very windy and that makes the atmosphere more disturbed. But the
>> > night sky here is spectacular.
>> >
>> > Another nice thing here is that up here there is a small animal
>> > called Marmote. I've never seen one, even in zoo, and on the
>> drive
>> > up we could see plenty of them. These are very shy animals, so
>> > they run away as we came driving up. However, occasionally they
>> > come up to the observatory. Just the other day, one Mermote
>> walked
>> > over the observatory keepers foot, not run, but walked. So, we
>> > joked that the Marmotes do not fear the scientists. :)
>> >
>> > We where able to come up with the clocks a day early, so we
>> got to
>> > interact with the previous mission. Friendly and hospital
>> people.
>> > So they shared the reminder of their dinner food, cheese and
>> wine
>> > with us. The hobbyist astronomers can apply for one-week
>> missions
>> > to Astro Queyras that. Depending on the quality of the mission,
>> > they get access to the site for a week, and good mission
>> gets the
>> > good weeks, with minimal moon light polution. Normally it is
>> full
>> > isolation, but gaining another day of integrating up the
>> frequency
>> > error they bent the rules for us, and me and Bruno could come
>> > early. We are very happy as we also got to interact with the
>> > preivous mission here.
>> >
>> > If there is interest, I can keep sending updates. Yesterday was
>> > hectic, so it was not the time to pull out data. That work
>> we can
>> > start doing today. I will also do a write-up of the setup if
>> > people are interested.
>> >
>> > So, I was given the oppertunity to do this with very short
>> notice,
>> > and I jumped on the oppertunity. It's been very stressful
>> and hard
>> > work to come to this point, but now I can start to enjoy it.
>> Also,
>> > time to share the experience.
>> >
>> > Today will the rest of the total mission come up, I only work on
>> > the graviational red-shift mission, but the full mission
>> involves
>> > astronomy missions naturally, such as spectroskopy. There is
>> also
>> > a documentation side, so things is filmed etc.
>> >
>> > Cheers,
>> > Magnus
>> >
>> > On 2023-07-16 06:48, Christopher Hoover wrote:
>> >> Best of luck, Magnus.
>> >>
>> >> On Fri, Jul 14, 2023 at 4:04 PM Magnus Danielson via time-nuts
>> >> <time-nuts at lists.febo.com> wrote:
>> >>
>> >> Fellow time-nuts,
>> >>
>> >> So, 5 years ago I was asked by Observatoir Royal de
>> Belgique
>> >> to take
>> >> care of two failed 5071A cesiums and alter them so that
>> there
>> >> is one
>> >> functioning. I did that, and have operated the functioning
>> >> one here and
>> >> there, but conservatively, since they wanted to do a
>> mission
>> >> with
>> >> graviational red-shift.
>> >>
>> >> Since them, other things happen in life, such as Covid,
>> >> prohibiting
>> >> work. However, a few weeks ago I was contacted as they
>> where
>> >> about to go
>> >> on the mission. I thought it would be nice to join, as
>> I had
>> >> quite a bit
>> >> of the things needed for such mission.
>> >>
>> >> Setting it up meeds a number of challenges, as it needs
>> to be
>> >> powered
>> >> continuously, and over a varity of sources.
>> >>
>> >> In addition I wanted to log as much data as possible. I've
>> >> come far on
>> >> that point. I log 5071A state and several other environment
>> >> sensors into
>> >> an InfluxDB and then illustrate with Grafana. Independently
>> >> two GNSS
>> >> receivers log things for post-processing.
>> >>
>> >> Just assembling the rig for the two 100 W PV panels and a
>> >> choke-ring
>> >> antenna has been a challenge. Let's just say that I should
>> >> not be hired
>> >> to do fine mechanics work.
>> >>
>> >> Things have been fixed during the travel, and I have driven
>> >> 600+1100+800
>> >> km just to be in base-camp before final climb. I'm now
>> >> sitting in a
>> >> borrowed house just outside of Grenoble. I just had to tear
>> >> out part of
>> >> the equipment, as one of the power converters failed.
>> Luckily
>> >> that side
>> >> was redundant for the mission, but I do miss one pressure
>> >> sensor to
>> >> compare the small one with.
>> >>
>> >> I had intended for my passive hydrogen maser to also do the
>> >> trip, but it
>> >> woke up with an unexpected problem and did not lock, and I
>> >> decided it
>> >> was not meaningful to bring it's dead weight along. I will
>> >> have to
>> >> investigate the actual fault. Also, I have been unable
>> to log
>> >> the
>> >> internal state, which Would have given valuable clues.
>> >>
>> >> Tomorrow we do the final climb to Observatoir de St
>> Veran at
>> >> 2930 m and
>> >> stay there for 8 days.
>> >>
>> >> Connectivity can be sketchy.
>> >>
>> >> In the end of the day, it is only if you try that you can
>> >> either fail or
>> >> succeed. So far, it has not yet failed completely
>> >> catastrofically, but
>> >> warnings-signs has gone of and they been managed, so far.
>> >> This can still
>> >> fail spectacular.
>> >>
>> >> Now I badly need my bed. Being sleep deprived does not help
>> >> thinking.
>> >>
>> >> Cheers,
>> >> Magnus
>> >> _______________________________________________
>> >> time-nuts mailing list -- time-nuts at lists.febo.com
>> >> To unsubscribe send an email to
>> time-nuts-leave at lists.febo.com
>> >>
>> _______________________________________________
>> time-nuts mailing list -- time-nuts at lists.febo.com
>> To unsubscribe send an email to time-nuts-leave at lists.febo.com
>>
>> --
>> Jim WN1X
> _______________________________________________
> time-nuts mailing list -- time-nuts at lists.febo.com
> To unsubscribe send an email to time-nuts-leave at lists.febo.com
More information about the Time-nuts_lists.febo.com
mailing list