User Tools
Sidebar
operations:handover
This wiki is not maintained! Do not use this when setting up AuScope experiments!
Handover notes
Please use this page to write up any notes on anything that needs to be passed on to the next observer or should go into the end of experiment message.
Notes should be placed below the horizontal line as a new blog entry.
Continuing problems should be copied to the Current Issues page.
**Template for Handover notes:**
Be sure to add any specific notes about the experiment if required, for example “Special back end configuration for x reason”. Put as much detail as you can - the more the better.
It is ESSENTIAL to put your name after EVERY entry.
Copy and paste the template into the new handover notes
====== Experiment Name ====== **Hobart 12m** Disk VSN: abc-123 | Data volume at beginning: 2.34 GB * 1300UT a comment from me (user) **Katherine 12m** Disk VSN: | Data volume at beginning: GB * **Yarragadee 12m** Disk VSN: | Data volume at beginning: GB * 1300UT Experiment started okay (JMc).
Format for entering a comment It is important to stick to this formula - there is a good reason we need this information, so please include it!
Time stamp in UT * Comment * (Your Name)
For Example:
20:00UT: Wind stows, Missed scans 012-3456 to 012-3457 (Jim)
20:50UT: Mark5 lost connection, had to restart Mark5 and reconfigure DBBC, called Jamie on call to fix, missed scans 012-3456 to 012-4567 (Jim)
Again - as much detail as you can. It is important to remember that these notes will be sent to the correlator, so please make sure they make sense to others.
r1781
Hobart 12m
Module HOB+0069, used 0.0 GB
- 1300UT Multiple instances of dimino running, cannot be terminated using EndDIM; mark5 rebooted (Ross)
- 1345UT Cannot record to module OAN+0103, rebooted mark5 again (Ross)
- checkmk5 gives: 2017.058.13:52:14.70/mk5/!get_stats? 0 : 3 : 293 : 6 : 5 : 14 : 26 : 8 : 10 : 10 : 0 : Fault ; (Ross)
- 1410UT Need to go to Mt Pleasant to change module, BUT THERE ARE NO CARS!!! (Ross)
- 1500UT Walked to parents house and borrowed car, now at Mt Pleasant. Disk is faulty, replaced with new module (Ross)
- 1510UT Autocorrelation spectra now not sensible. Restarting dimino (Ross)
- 1515UT Seems to all be working now, acs problem seems to be related to testing other module. Have completely removed this now (Ross)
- 1700UT Experiment started OK (Ross)
Katherine 12m
Module HAY-0062, used 2.564 GB
- 1700UT Experiment started OK (Ross)
Yaragadee 12m
Module UVLBI-27, used 0.0 GB
- 1700UT Experiment started OK (Ross)
R4780
Hobart 12m
Module HOB+0042, used 3363.39 GB
I am using this module, since there are no 'allocations' and it has about the right amount of space
- 1830UT Experiment started OK (JS)
Katherine 12m
Module USN-0145, used 0.0 GB (offset 577.5 GB)
- 0200UT Experiment started OK after dbbc repaired, first scan 055-0207a (Ross)
- Halted the experiment at one point to reconfigure the dbbc because there were no system temps.
Yaragadee 12m
Module USN+0221, used 0.0 GB
- 1830UT Experiment started OK (JS)
- 0517UT VPN needed resetting. (Arwin)
ohg108
Hobart 12m:
Disk VSN: HOB+0059
- 17:30UT Exp started OK. First source 1334-127. (Arwin)
- 22:18UT missed a scan due to dbbc resync (power cycled wrong one), there's a small clock jump (-0.004 usec). Missed scan 053-2217 (Bryn)
- 10:30UT halted for spaceX tracking. Missed scans 054-1030, 054-1030, 054-1037, 054-1041a and 054-1055. (Jonny)
Data volume at beginning: 0 GB
Katherine 12m:
Disk VSN: BKG-0015
Data volume at beginning: 0 GB
- 18:01UT Exp started late, issues with the DBBC and the schedule. First scan is 053-1801, source 0727-115. (Arwin)
- 19:18UT Missed scan 053-1918 to reconfigure DBBC. The comments.py should auto pick this. (Arwin)
- 21:30UT Antenna got stuck, released at 22:05UT with antenna=open and antenna=operate. Scans affected are 053-2130 through 053-2204 (inclusive) (Bryn)
- 22:15UT delay drifting, resetting DBBC missed scans 053-2215 through 053-2222.
- 00:50UT Bad dbbc delay. Delay difference seems okay - not drifting, but if the alarms continue, I'll reset the DBBC again. (Bryn)
- 01:25UT resetting dbbc. Missed scans 054-0125b and 054-0129 (Bryn)
- 01:29UT Problems with dbbc, halting schedule (Bryn)
- 02:25UT Have tried full power cycling dbbc several times, still can't get consistent clkoff from machine - fmset no use. Talked to Jim, Warren and Brett - stowing telescope and keeping schedule halted. Can't retrieve system temperatures, so no point continuing until issue is resolved. (Bryn)
Yarragadee 12m:
Disk VSN: HOB+0041
Data volume at beginning: 0 GB
- 18:31UT Exp started late, issues with the mk5 and schedule.First scan 053-1831 and source 1057-797. Mk5 was unresponsive for while and the later it wouldn't record data. Typing 'schedule=' would give a
no scanserror and the dish would find the source but not record. Spammed setupsx and schedule=, it came right on the third or so attempt. (Arwin)
r1780
Hobart
Disk VSN: HOB+0042
Volume at Beginning: 0GB
- Can't connect with eremote control, Dave's backup isn't working either - doing setup with VNC. (Bryn)
- 1700UT Experiment started OK. (Bryn)
- 2210UT Tiege taking over, current telescope status: telescope working OK. (Tiege)
- 0946UT Halt issued for SPACEX tracking pass(JS)
- 1021UT Onsource after SPACEX (JS)
- 11:23UT SpaceX started and stopped at 12:02UT. The next recording was 12:08UT. Jon noted that the pointed corrections were set to REFRAC+MODEL ever since the last spacex run, currently set to NONE.
- 12:57UT halted for Spacex again. (Arwin)
- 13:38:30 UT recording started again after spacex. (Arwin)
Katherine
Disk VSN: CURT+102
Volume at Beginning: 0GB
- 1646UT Autocorrelations are bad, iread is giving values severely under target levels. Tried resetting the DBBC (Bryn)
- 1700UT Tried resetting and reconfiguring DBBC, but the IFs are still way under target values. (Bryn)
- 1720UT ifnom doesn't fix the IFs, no point starting experiment without fixing, as system temperatures are awful. (Bryn)
- 2210UT Tiege taking over, current telescope status: Currently not observing, Telescope IFs/power levels broken. Bryn said Warren had a go at fixing. Guess I will take another look at it and see if I can get Ke on schedule and observing. (Tiege)
- 2247UT Still trying to fix this, when I try and restart Monica.IFBox I get 'Socket created', then 'Bind failed'. Probably has something to do with it. DISREGARD THIS, worked second try. (Tiege)
- 2225UT On the Phone to Brett, trying to fix this telescope. Bretty seems to think that something might be off in the telescope pedestal, meaning that the receiver is powered down. Martin is going to go take a look in 20 mins or so. (Tiege)
- 2350UT IF power level issue fixed. All I ended up having to do was use the 'reboot system' command on the UoT interface on the timing computer. Ke has now joined R1780, did not manage to complete a pointing check for it as I got error fivept command not setup correctly. Apart from that it seems to be recording fine now. First valid scan is 052-2356a. (Tiege)
- 0011UT Field system complaining about antenna pointing, interrupst 053-0009 to run a pointing check. this was completed by 0020 UT and first valid scan since is 053-0026. (Tiege)
- 0040UT The problems continue, now getting rfpcn:time-out errors which have been seen many times before. Tend to be fixed by restarting Rxmon on fs pc, halted schedule and restarted Rxmon. First scan after this reset is 053-0047b (Only missed 053-0040a). (Tiege)
- 0140UT rfpcn time-out error happening again, restarted Rxmon again. will keep monitoring. (Tiege)
- 15:51 UT noticed that the generator state is not green. (Arwin)
Yarragadee
Disk VSN: USN-0093
Volume at Beginning: 0GB
- Getting lots of errors with the mk5 during setup, probably due to issue with counter (can't run clkoff). Have tried using 'counter' to reset counters, still getting errors as follows:
- WARNING: error sc -13 setcl: formatter to fs time difference 0.5 seconds or greater
- WARNING: error sc -23 setcl: mark 5b not sync'd, consider using fmset 's' option to fix.
- WARNING: error ib -4 gpib device time-out on response c1
- ALARM: error m5 -104 mk5cn: time-out, connection closed
- WARNING: error m5 -112 mk5cn: eof pre-draining input, mark 5 may have crashed, connection closed
- WARNING: error m5 -114 mk5cn: re-open after pre-drain error was okay, proceeding to transaction.
- Test recording has times way off, but fmset won't run because of the above errors. Have tried restarting the mk5 - no changes. (Bryn)
- mk5 records fine, mk5=dot reports FHG_on. However, running scan_check causes ERROR m5 -900 Can't XLRRead() (Bryn)
- Tried physical reboot of system by calling on-site person. Same errors persist. (Bryn)
- Apparently some work was done on the wiring of Yg since the last experiment. This might be the source of the issue. If so, there isn't anything I can do remotely. (Bryn)
- Clkoff is working again, after having a manual reset by on-site personnel. Time is not stable, still unable to run fmset. (Bryn)
- 1700 UT Experiment on hold - still investigating. (Bryn)
- 2210UT Tiege taking over, current telescope status: Experiment still on hold, still cannot connect to mk5. Going to take another look at this and see if I can sort it out. (Tiege)
- 2325UT Currently on the phone to brett attempting to fix Yg and Ke, On-site people at Yg are taking a look at the telescope for us while we look at Ke. (Tiege)
- 0031UT After many calls with Brett and Randall it seems that Yg may be finally joining R1780. Manual reboot of the DBBC pc fixed some issues, but it still did not come up all peachy. Reconfiguring the DBBC twice, restting the counters with 'counter' and then running fmset seemed to clear up all the issues. Ran a pointing check and the antenna is onsource with the first planned scan being 053-0040a (Tiege)
- 0046UT First scan on Yg complete, everything looks OK. (Tiege)
rd1703
Hobart 26m
Disk VSN: HOB+1003
Data volume at beginning: 1.433 GB
- 14:30UT mkv could not be contacted and needed a reboot. Updated wiki with reboot instructions. (Arwin)
- 16:33UT Exp started OK and on time. (Arwin)
- 16:33UT The sum file is called rd1703ho but the scans are for rd1702ho? The fs system is getting the correct scans and source, I checked this in the log
scan_name=051-1633,rd1703,ho,40,40and2017.051.16:15:55.77:source=0700-197,070032.35,-194655.8,1950.(Arwin) - 08:30UT The Vacuum pressure is 3.8 and 70K stage at 75. Consulted with Brett, he recommends keeping an eye and calling him if it doesnt come down. (Arwin)
- 10:10UY The Vacuum pressure has dropped to 2.8. The 70K stage also drooping slowly, currently at 62.8, down from 65. (Arwin)
r4779
Hobart 12m:
Disk VSN: HART+101
Data volume at beginning: 3228.254
- 18:00UT Exp started OK. (Jonathan)
Katherine 12m:
Disk VSN: FGI-0005
Data volume at beginning: 1089.824
- 18:00UT Exp started OK. (Jonathan)
- 1427 Antenna stuck, restarting drives (JS)
- 1432 Fixed, onsource and tracking (JS)
- Scans affected 048-1430
- 1441 Halting schedule for rfpcn timeout and restarting DIMino (JS)
- 1445 Resumed schedule fro 048-1445b (JS) [Seems to be fixed now]
- Scans affected 048-1442
- 1459 Halt and restart rxmon/monica to fix rfpcn time-out (JS)
- resumed at 1503:50 affected start of scan 048-1503
Yarragadee 12m:
Disk VSN: USN-0140
Data volume at beginning: 0 GB
- 18:00UT Exp started one scan late. (Jonathan)
/home/www/auscope/opswiki/data/pages/operations/handover.txt · Last modified: 2018/10/29 16:27 by Jesse Swan
