User Tools
Sidebar
This wiki is not maintained! Do not use this when setting up AuScope experiments!
Handover notes
Please use this page to write up any notes on anything that needs to be passed on to the next observer or should go into the end of experiment message.
Notes should be placed below the horizontal line as a new blog entry.
Continuing problems should be copied to the Current Issues page.
**Template for Handover notes:**
Be sure to add any specific notes about the experiment if required, for example “Special back end configuration for x reason”. Put as much detail as you can - the more the better.
====== Experiment Name ====== **Hobart 12m** Disk VSN: abc-123 | Data volume at beginning: 2.34 GB * 1300UT a comment from me (user) **Katherine 12m** Disk VSN: | Data volume at beginning: GB * **Yarragadee 12m** Disk VSN: | Data volume at beginning: GB * 1300UT Experiment started okay (JMc).
Format for entering a comment It is important to stick to this formula - there is a good reason we need this information, so please include it!
Time stamp in UT * Comment * (Your Name)
For Example:
20:00UT: Wind stows, Missed scans 012-3456 to 012-3457 (Jim)
20:50UT: Mark5 lost connection, had to restart Mark5 and reconfigure DBBC, called Jamie on call to fix, missed scans 012-3456 to 012-4567 (Jim)
Again - as much detail as you can. It is important to remember that these notes will be sent to the correlator, so please make sure they make sense to others.
CN1603
KATHERINE
Disk Module: BKG-0062
Start Disk Poisition: 0.705 GBs
- 18:10 UT Experiment started late and schedule file would not load. Re-drudged and schedule worked. Missed scans 335-1800. 335-1808. First scan 335-1816 (Lucas).
- 07:54 UT - Alarm reporting illegal state with the drives. Also gave an error reading the system clock. “source=disable” and “antenna=open” seemed to fix it. (Ellen)
- 07:59 UT - Alarm reporting “error s5 -104 rfpcn: time-out, connection closed”. Restarted rxmon, seems to have fixed it. Scans reporting error 336-0759 and 336-0803. (Ellen)
YARRAGADEE
Disk Module:USN-0131
Start Disk Poisition: 0705 GBs
- 18:00 UT Experiment started okay (Lucas).
- 06:18 UT - Noticed that we're now recording to USN-0198. Checking logs and 741 GB was recorded before the module swapped at 02:33 UT. (Ellen)
rd1611
Hobart 26m:
Disk VSN: HAY-0055
Data volume at beginning:1.2 GBs
- 17:34UT Experiment started okay (Liz)
- 17:40UT ERROR qi -302 Last commanded IFD/IF3 attenuations used as default, please modify command. Followed wiki instructions to change attenuation, seem to solve error. (Liz)
- 17:50UT ERROR 5k -602 SCAN_CHECK missing bytes is not zero. Can't find reference to this on the wiki. No major alarm sounded so feel safe to wait until Jamie takes over. Non-critical? (Liz)
- 19:40UT System temps have been on the high side ~100 across all bands on and off. Some instances of major overflow in a number of bands but also long periods where they remain in an acceptable range. Monitoring for now. (Liz)
- 21:30UT turns out system temps were high because the receiver was still on l band. now fixed (Liz)
- 21:30UT ERROR 5k -602 SCAN_CHECK missing bytes is not zero. THIS IS NOT GOOD. Jamie is trying to fix now. (Liz)
- 22:00-22:45 Attempted to use jive5ab 7 in2file to fix the problem, but m5test indicates the same problem in the data. Warren and Brett now investigating the connections between the mark4 rack and mark5a unit (JMc)
- 23:00 No change, scan_check still reports dropped bytes. m5test indicates occasional loss or time codes/sync in recorded data using both disk_record and in2file. Continuing experiment with disk recording (JMc).
r1768
Setup Notes
- Had a few network issues where the commands wouldn't register, lasted about 30 minutes. Ke and Yg ercs kept failing.
- Ke had a “ERRORdb -888 DBBC PPS error”. No mention if it on the wiki, dbbc software on dbbcke vnc was displaying things slightly different from dbbchb and dbbcyg. Reconfigured the dbbcke, and ran a fmset, this seems to have fixed the error
- Missed out on fringe checks, as I ran out of setup due to a earlier experiment involving hb and network isses.
- Can't ssh into ops2, have tried restarting op2, no luck. As a result no emails can be sent. Manually emailed Jim, Jamie, Lucia etc telling them that exp has started.
Hobart 12m:
Disk VSN: HOB+0130
Data volume at beginning: 0.577 GB
- 1700UT Experiment started OK. (Arwin)
Katherine 12m:
Disk VSN: HOB+0105
Data volume at beginning: 5213.448 GB
- 1700UT Experiment started OK. (Arwin)
Yarragadee 12m:
Disk VSN: HOB+0123
Data volume at beginning: 4774.988 GB
- 1700UT Experiment started OK. (Arwin)
aug033
Hobart 12m:
Disk VSN:HOB+0051
Data volume at beginning:0.0 gb
- 19:30UT Experiment started OK.
- 19:35 UT got the following errors “ERROR st -998 reading SystemClock1”, “ERROR st -999 TCP/IP connection error” , and “ERROR st -5 Error return from antenna, see Mbus error.”. antenna=open didnt fix, neither did restarting the HMI, or the field system. Warren heading to the observatory to investigate. Scans affected 329-1937 to 329-2101 inclusive. (Arwin)
- 19:45 UT Network problem, syshb.phys.utas.edu.au (131.217.63.148) wasn't responding to packets, restarted network switch in 12m pedestal. (Warren)
Katherine 12m:
Disk VSN:GSFC+018
Data volume at beginning: 0.0 gb
- 19:30UT Experiment started OK.
- 02:40UT mk5 died. Possibly Martin swapped the module. Killed the dimino and restarted dimino after running SSReset. Missed scans: 330-0231 at 02:31 through 330-0241 first good scan 330-0244 at 02:44. (Lucia)
- 02:50UT e-remote control is running again now. Jamie killed the ercd processes on pcfske and restarted manually. (Lucia)
Yarragadee 12m:
Disk VSN:USN-0097
Data volume at beginning: 0.0 gb
- 19:30UT Experiment started OK.
Hobart 26m:
Disk VSN:USN+0221
Data volume at beginning: 0.0 gb 1144-379
- 19:33UT Experiment started 3 minutes late due to delay in setup resulting from issues with ke eremotecontrol. First scan 329-1939, source= 1144-379.
- 05:48UT Alarm source not reached in time. There was a wind stow. Affected scans between 5:44 and 05:49 UT. (Lucia)
- 06:42UT - Wind stowed for scans 330-0644 to 330-0708. Still too windy so schedule was halted because alarm sounds every single scan. Missed scans 330-0711 to 330-0806 (80 GB).(Ellen)
- 08:08UT - Wind settled down. First scan back 330-0811. (Ellen)
- 17:04UT Alarm sounded,“Previous source in this schedule not reached before new source was commanded” doesn't seem to be because of wind stow or any other system error. Next source was reached on time, and is now tracking. Affected scan 330-1704. Will monitor for recurrence and possible causes. (Liz)
- 18:59UT “Previous source in this schedule not reached before new source was commanded” happened again. This time, live page indicates wind stow. Waiting for self-recovery, however, doesn't seem to be windy enough to stow. Checking wiki but may run out of time on the experiment (Liz)
- 19:31Ut According to Jamie, it has been windy enough to stow so this is nothing to worry about. Missed scans 330-1858 to 330-1932 (Liz)
CRDS86
Hobart 12m:
Disk VSN: MED-0038
Data volume at beginning:0.321 GBs
- Experiment Started Okay (Lucas).
- 22:56 UT - Schedule halted for cable upgrades by Jim L, last scan recorded before halting 328-2250. (Ellen)
- 04:28 UT - Schedule resumed. First scan back 329-0437. Missed scans 328-2303 to 329-0424, inclusive. (Ellen)
Katherine 12m:
Disk VSN: WSRT-061
Data volume at beginning:0.321 GBs
- Experiment Started Okay (Lucas).
Yarragadee 12m:
Disk VSN:USN-0062
Data volume at beginning: 0.288 GBs
- Experiment Started Okay (Lucas).
Hobart 26m
Disk VSN: HOB+1006
Data volume at beginning: 1776.114 GBs
- Experiment started late as mkv crashed and was unable to ping. No IP switch. Warren was able to re-set mk5a rack at Mt P.
- 20:05 UT, experiment started. First source 328-2004 (Lucas).
- 21:00 UT mk4 rack time wrong, reset; first recording 328-2109 source 0549-575, start data 1815 GB, missing ~150 GB (Warren)
- 22:05 UT - Receiver is off by a couple of mm, had the same problem in a previous experiment after Jim P said he could not get the L-band receiver on axis. Lucas (?) said it was fine though. (Ellen)
- 23:14 UT - Massive negative delay difference, but nothing wrong following scans. (Ellen)
- 23:50 UT - Tsys values consistently bad in bands 10-14. (Ellen)
- 01:31 UT - error m5 -104 mk5cn: time-out, connection closed. Mk5 crashed, no longer responding to anything. Warren had to hard reset. Only bad scan 329-0131, schedule resumed straight after. (Ellen)
- 01:50 UT - Tsys bands 13,14 still bad, other bands are now fine. (Ellen)
r4767
Hobart
Module: HOB+0047, start pos 0.577 GB
- 1830UT - Experiment started OK. (Bryn)
Katherine
Module: USN-0080, start pos 0.417 GB
- @mondump is causing a huge amount of output - could this lock up the field system communicating with the drives?
- 1830UT - Experiment started OK. (Bryn)
- 1906UT - Antenna got stuck, didn't notice because no major alarms sounded (Bryn)
- 2110UT - Issue with drives resolved - scans 327-1906a to 327-2118a (Bryn)
Yarragadee
Module: MAT+0010, start pos 0.417 GB
- 1830UT - Experiment started OK. (Bryn)
