User Tools
Sidebar
This wiki is not maintained! Do not use this when setting up AuScope experiments!
Handover notes
Please use this page to write up any notes on anything that needs to be passed on to the next observer or should go into the end of experiment message.
Notes should be placed below the horizontal line as a new blog entry.
Continuing problems should be copied to the Current Issues page.
**Template for Handover notes:**
Be sure to add any specific notes about the experiment if required, for example “Special back end configuration for x reason”. Put as much detail as you can - the more the better.
====== Experiment Name ====== **Hobart 12m** Disk VSN: abc-123 | Data volume at beginning: 2.34 GB * 1300UT a comment from me (user) **Katherine 12m** Disk VSN: | Data volume at beginning: GB * **Yarragadee 12m** Disk VSN: | Data volume at beginning: GB * 1300UT Experiment started okay (JMc).
Format for entering a comment It is important to stick to this formula - there is a good reason we need this information, so please include it!
Time stamp in UT * Comment * (Your Name)
For Example:
20:00UT: Wind stows, Missed scans 012-3456 to 012-3457 (Jim)
20:50UT: Mark5 lost connection, had to restart Mark5 and reconfigure DBBC, called Jamie on call to fix, missed scans 012-3456 to 012-4567 (Jim)
Again - as much detail as you can. It is important to remember that these notes will be sent to the correlator, so please make sure they make sense to others.
r1896
Katherine 12m
Disk VSN: GSFC+024 | Data volume at beginning 0.769 GB
- 17:00 UT - Experiment started okay (Momtaz).
- 17:30 UT - FS crashed. Schedule restarted at 17.35 UT(Momtaz).
- 08:37 UT - FS crashed. Schedule restarted at 08:50 UT. Scan 149-0838b, 149-0843, 149-0847 probably missed. (Lim).
- 10:09 UT - FS crashed. Schedule restarted at 10:13 UT. Missed scan 149-1005, 149-1010a. Scan 149-1014 and 149-1017 probably missed (Lim).
- FS Crashed (unsure when, no alarm). Schedule restarted at 14:21 UT, with scan 149-1423. (Earl)
- 14:30:32 ALARM: scan_check reports Mark5 data format problem (Earl)
- 14:33:24 ALARM: error m5 -900 can't read while data transfer in progress (Earl)
r4895
Katherine 12m
Disk VSN: TR-00022| Data volume at beginning: 0.0 GB
- 1830 UT - Experiment started okay (Lim).
- 1911 UT - “Resources temporarily unavailable quikv terminated-Exit code 225”. Not sure what happened, the field system, log, proc were all terminated. The problem had gone unnoticed until alarm gone off at 1929 UT showing no pcfs activities for the last 15 minutes. All scans between 143-1911 and 143-1944b missed (Lim).
- 22:06 UT problem above happened again. Noticed and solved at 22:15 UT. Might have missed scan 143-2206a, 143-2208 and 143-2213. Tsys has died (Lim).
- 0310UT - FS crashed. Schedule restarted at 0323UT (Prad).
- 0345UT - FS crashed. Schedule restarted at 0359UT (Prad).
- 15:50UT- FS crashed. Schedule restarted at 16.04UT (Momtaz)
T2132
Katherine 12m
Disk VSN: HOB+0093
* 17:00 UT Experiment started OK (Earl)
Hobart 26m
Disk VSN: HOB+0125
* 17:00 UT Experiment started OK (Earl)
R1895
Katherine 12m
Disk VSN: HOB+0093 | Data volume at beginning: 10.25 TB
- 17:00 UT Experiment started okay (Jay)
- 21:05 UT Field system crashed( Don't know why it crashed, maybe some network issue). Restarted field system, did quick setup. missed scans from 140-1945 to 140-2101. (Jay)
- 23:49 FS crashed again - terminated with error “cls_rcv: sending saved message: Resource temporarily unavailable” followed by “quikv teminated-Exit 255” and then the usual shutdown messages. Restarted the FS at 00:08, schedule resuming with first scan at 00:18. No useful data between 23:49 and 00:18 (JMc).
- 0515, another FS crash, same symptoms. Rebooting pcfske (JMc)
- Resumed with first good data at 0540. No useful data between 0515 and 0540. (JMc)
- 12:23 UT Field system crashed. Missed scans - 141-1223 until 141-1249b. (Katie)
- 15:00 UT (approx.) FS Ke crashed. (Earl)
Yarragadee 12m
Disk VSN: USN-0005 | Data volume at beginning: 0.7 GB
- 17:00 UT Experiment started okay (Jay)
R4894
Katherine 12m
Disk VSN: BKG+0138 | Data volume at beginning: 0 GB
- 18:30 UT Experiment started OK by Lucas (Warren)
- 01:00 UT Wrote VSN to module (w)
Yarragadee 12m
Disk VSN: USN-0201 | Data volume at beginning: 0 GB
- 18:30 UT Experiment started okay by Lucas (Warren)
- 01:00 UT Wrote VSN to module (w)
AUA052
Hobart 26m
Disk VSN: HOB+0125/32000/1024 | Data volume at beginning: 18.45 TB
- 1800UT - Experiment started ok(Jay).
- 1400UT - Disk position is significantly offset, around 1TB! Why has noone put this in the handover notes? The decrease seems steady over the experiment:
Katherine 12m
Disk VSN: HOB+0093/16000/1024| Data volume at beginning: 0.0 GB
- 1800UT - Experiment started OK (Jay)
Yarragadee 12m
Disk VSN: HOB+0131 | Data volume at beginning: 5.9 TB
- 1800UT - Experiment started OK (Jay)
- 136.14:45:00 UT - connection lost to pcfsyg. Cannot vnc or ssh into system, ping times out. Could be power failure. Will attempt to contact SNR station (Lucas)
- Phone lines worked, was able to call Yarragadee and Peter said everything appeared to be working correctly (Lucas)
- Called Brett:
- Network router at Yarragadee is 131.217.61.65. Pinged this and it timed out. Implying that the network router was down (Lucas).
- Turns out the on-site router was in a weird state. ROUTER MSR2004-24 JG734A, second unit from the top of rack 4. Required a power cycle. It has 2 ethernet ends and 1 mobile. One through the ambulance link and the other through the ming windmill link (?). Unknown where the problem was but after the reboot it came back up (Lucas).
- 16:10:00 Once network was back up, was able to see that antenna had lost communication and required a antenna=open, antenna=operate. Logs reveal this was only from 16:03:00 so only affecting scans 1600 to 1610 (Lucas).
