User Tools

Site Tools


operations:handover

This wiki is not maintained! Do not use this when setting up AuScope experiments!

Handover notes

Please use this page to write up any notes on anything that needs to be passed on to the next observer or should go into the end of experiment message.

Notes should be placed below the horizontal line as a new blog entry.

Continuing problems should be copied to the Current Issues page.

**Template for Handover notes:**

Be sure to add any specific notes about the experiment if required, for example “Special back end configuration for x reason”. Put as much detail as you can - the more the better.

It is ESSENTIAL to put your name after EVERY entry. Copy and paste the template into the new handover notes

====== Experiment Name ======

**Hobart 12m**

Disk VSN: abc-123 | Data volume at beginning: 2.34 GB

   * 1300UT a comment from me (user)

**Katherine 12m**

Disk VSN: | Data volume at beginning:  GB

   * 

**Yarragadee 12m**

Disk VSN: | Data volume at beginning:  GB

   * 1300UT Experiment started okay (JMc).

Format for entering a comment It is important to stick to this formula - there is a good reason we need this information, so please include it!

Time stamp in UT * Comment * (Your Name)

For Example:

20:00UT: Wind stows, Missed scans 012-3456 to 012-3457 (Jim)

20:50UT: Mark5 lost connection, had to restart Mark5 and reconfigure DBBC, called Jamie on call to fix, missed scans 012-3456 to 012-4567 (Jim)

Again - as much detail as you can. It is important to remember that these notes will be sent to the correlator, so please make sure they make sense to others.


A1501

Hobart 12m:

Disk VSN: HOB+0127

Data volume at beginning: 0 GB

  • Experiment started ok.
  • 23:00UT: Sharp spike in band 13 autocorrelation function (Ross).
  • 12:38 UT - disk_pos is ~70 gb short (vk)
  • 13:52 UT - ~100 GB short (vk)

Katherine 12m:

Disk VSN: HOB+0129

Data volume at beginning: 0 GB. Now -50GB due to late start

  • Experiment started 20 min late. Clock offset were heavily drifting. Had to restart the DBBC. Look good now (Lucia)
  • 12:38 UT - disk_pos is ~70 gb short (vk)
  • 13:35 UT - disk_pos is ~70 gb short (vk)

Yarragadee 12m:

Disk VSN: HOB+0123

Data volume at beginning: 0 GB

  • Experiment started ok.
  • 17:15UT: Yg called and reported a power failure, they run on the generator for the next ~1h. Observing look fine (Lucia)
  • Problems with Tsys readings as usual.
  • 23:00UT: Upon typing clkoff and maserdelay commands, delay difference worked out as 10^6 seconds. Repeated and sensible but this problem has occurred before (Ross).
  • 23:15UT: Disk_pos 100GB behind schedule (Ross).
  • 01:08UT: Disk_pos 135GB behind schedule, Jamie fixed issue with delayed start of recording (Ross).
  • 1028UT: Antenna connection crashed, reopened at 1034UT (JS)
  • 1238 UT - It looks like internet connection to Yg has fallen out. Cannot seem to call them even. Will try pining in a few minutes.
  • 13:53 UT - Internet connection re-established. D-link modem restarted. Disk_pos is now ~200 behind prediction (vk)

Hart 15m:

  • ERC couldn't get connection. Monitoring since 18UT.
2015/02/03 05:41 · 0 Comments
AUG007

No Katherine again.

Hb

Module: HOB+0126, start pos 18594 GB

  • Experiment started ok, went straight to the next schedule. (Ellen)

Yg

Module: HOB+0097, start pos 4817 GB

  • Experiment started ok, alarms went off because it didn't go straight to the next schedule, but no scans were missed cos I was super fast. (Ellen)
  • 2015.032.08:00:33 wind stow, 08:05:39 halt schedule (warren)
  • 08:33 field system restart to get out of wind stow and back on schedule and recording for scan 032-0838 (warren)
  • 17:46 UT: Antenna reported “stuck”. antenna={open,operate} did not restore initially. After rebooting central, the connection was restored. 032-1738 – 032-1743 missed (Dave)
  • 19:30 UT: Power outage at site, now on generators. Had to re-establish antenna connection, but otherwise everything seems to have switched fine. (Dave)
2015/02/01 00:01 · 0 Comments
AUG006,
or a story of Liza's tribulations on Saturday morning
  • As for 21:40UTC, network is not functioning at Sandy Bay campus. Unavailable servers: all remote servers. Unavailable services, related to the observations: slogit, ready-message and start-message script, log monitor.
  • 22:40UTC, after the tests that shown that Mt. Pleasant has network and all available computer on campus don't, going to Mt Pleasant to start observations. Drudging&editing&copying schedules by hand.
  • ~23:40UTC observations started. Log monitor is not functioning.
  • 02:00 UT: found a worker on campus who could forward me to the responsible UTAS guy. They didn't know about our problem but finally found out what's causing it. Weirdly, wireless connection works….. Anyway, they promise to fix it within 1h. That's why I decided to stay at campus and keep watching Hb and Yg via vnc through wireless on my laptop. (Lucia)
  • 03:30 UT: connection not back yet. All I can see through the vnc looks fine. (Lucia)
  • 04:15 UT: power outage finished, IT is working on network now. Antennas look fine (Lucia)
  • 05:45 UT: network finally back! Last hour also the wireless was problematic. Setup in AuScope room now (Lucia)

Hb

  • recording to module HOB+0126
  • First recorded scan:031-0040 (late start becvause of the network issues at Sandy Bay campus) (Liza)
  • RFI in last two channels all the time (Lucia)
  • 05:45 UT: pcfshb out of disk space. Can't write log file. This was going on probably for the last 30-60 minutes. All good now. (Lucia)

Ke

  • Katherine not observing due to DBBC problems

Yg

  • recording to module HOB+0097
  • First recorded scan:031-0047 (late start becvause of the network issues at Sandy Bay campus) (Liza)
  • 05:33 UT: wind stow - have to check when it started (Lucia)
  • 06:26 UT: antenna didn't release automatically from wind stow. Released manually. First scan after stow: 031-0632, @06:32UT (Lucia)
  • 06:38 UT: Tsys readings problematic due to DBBC problems (known problem). Many sband channels overflowed, xband channel 5 extremely high (1500). (Lucia)
  • 06:47 UT: automatic wind stow (Lucia)
  • 07:04 UT: first scan @ 07:08 UT (Lucia).
  • 07:36 UT: automatic wind stow. Last scan 031-0734b, the scan 031-0736 never started (Liza)
  • 07:43 UT: antenna released, first scan 031-0748 (liza)
  • 07:53 UT: windstow(Liza)
  • 08:25 UT: released from windstow, first scan 031-0831 (Liza)
  • 16:45 UT : windstow (Ellen)
  • 18:43 UT : first scan recording again after windstow. I can't find the schedule files so I don't know the exact scans missed. (Ellen)
2015/01/31 01:13 · 0 Comments
r4672

Hobart 12m: Disk VSN: HOB+0126, Data volume at beginning: 11,564.1 GB

Katherine 12m: Offline.

Yarragadee 12m: Disk VSN: USN-0182, Data volume at beginning: 266.6 GB

  • 2015.030.05:14:17 ALARM: error m5 -104 mk5cn: time-out, connection closed……….. Recording of scan 03-0514 seems to have continued despite this error, lost connection only briefly, or slow response from mk5 (warren)
  • Another instance at 07:35:05 (Chris)
  • (11:22) Scan CTA26 at 030-117 is missing data from 11:22:23, because the source was below the elevation limit. (Chris)
  • Another instance of mk5cn timeout at 11:55:14 (Chris)
  • UT 15:47 - Changed disk bank to HOB+0097/16000/1024 . First scan on new disk: 030-1548b (vk)
2015/01/29 17:36 · 0 Comments
CRDS75

Ho Module: JIVE-064, start pos 0 GB 4:44UT - small power glitch caused Mark 4 rack to reset some values, setupsx was run again and fixed the problem (Imogen)

Some how the log was set to station.log at 27.13:19, and most of the experiment was recorded to that log file. Copied out the appropriate for final log.

Hb Module: BKG+0133, start pos 0 GB

Ke Module:USN-0131, start pos 0 GB

DBBC not behaving, so couldn't observe (Ellen)

Yg Module: USN-0175, start pos 0 GB

Jamie tried to fix the power levels but couldn't. As a results, tsys bands 05 -12 are all bad. (Ellen)

2015/01/27 03:52 · 0 Comments
aug005

Hb Disk VSN: HOB+0126, Data volume at beginning: 5841 GB

00:00 UT - Experiment started OK (vk)

07:35 UT - Windstow (vk)

07:43 UT - Appears to have restarted on it's own w/o me being aware of it, as I was focused on starting up MoniCA. Estimated missed scans: 025-0734 to 025-0743.

23:45 UT - Disk_pos 80 GB short at end of experiment (Ross).

Ke Disk VSN: USN+0225, Data volume at beginning: 0 GB

00:00 UT - Experiment started OK (vk)

Yg Disk VSN: HOB+0033, Data volume at beginning: 0 GB

00:00 UT - Experiment started OK (vk)

Note: In the previous observation (aug004yg) the observer (Chris Jordan) noted difficulties in recording to bank A in mk5yg and therefore moved the module to bank B to record observations. Chris then warned that this issue might arise for aug005yg as well. However I have found no problem in recording to bank A at the start: disk_pos appears to be increasing as expected and autocorrelations are OK. (vk)

00:27 UT - As was the case in aug004 observations, we still have bogus values from iread: (vk)

2015.025.00:27:10.97/ifa/1,agc,1,25000,36,24653

2015.025.00:27:10.97/ifb/4,agc,4,29000,18,29254

2015.025.00:27:10.97/ifc/2,agc,2,35000,63,37290

2015.025.00:27:10.97/ifd/2,agc,2,35000,63,44086

08:45 UT - Yg disk_pos drifting. 50 GB short. 2 hours ago it was 20 GB short.

10:25 UT - Instance of “ALARM: Large difference between formatter and maser delays. Check for stability of new offset.” However, fmset showed no differences in the times and subsequent readings show normal ddif. Weird. (vk)

23:48 UT - Disk_pos 95 GB short at end of experiment (Ross).

2015/01/25 00:19 · 0 Comments
/home/www/auscope/opswiki/data/pages/operations/handover.txt · Last modified: 2018/10/29 16:27 by Jesse Swan