General Notes

All telescopes give WARNING: error bo -212 can't change experiment procedure library when an experiment procedure is executing when starting up dynamic schedule

Still regular slewing during recording at all telescopes, only a couple of seconds at the beginning though. (Ellen)

Halting the schedule to fix problems appears to have no effect (at least on Hb, Ke), the schedule continues trying to record. (Ellen)

Continuing to get “ALARM: error m5 -900 not while recording or playing” errors frequently at Ke throughout my shift, these have been reported by Bryn and Ellen as occurring all day, they don't seem to correspond to any actual problem with the telescope. (Tiege)

NOTE: I have changed this alarm m5 -900 to BEEP instead of ALARM in /usr2/control.ke/alarm_actions.txt so that Lucas can get some sleep between checklists

Hobart 12m

Disk VSN: GSFC+011

Data volume at beginning = 1829.6 GB

• 23:00 UT started recording on hb, halt didn't stop the schedule, so there will be 15 mins of recordings on hb. (Bryn)
• 23:25 UT enabled dynamic schedule (Bryn)
• 23:46 UT First good scan 288-2347 (Bryn)
• 01:23 UT Wind stow, missed scans 289-0122 to 289-0128
• 02:05 UT Wind stow, missed scans 289-0205 to 289-0209
• 03:30 UT Wind stow, missed scans 289-0330 to 289-0335
• 03:50 UT Wind stow, missed scans 289-0350 to 289-0411
• 04:35 UT Wind stow, missed scans 289-0435 to 289-0623
• 05:01 UT elevation drives not released when wind stow released. Immediately went into wind stow again - will check elevation drives when next released from wind stow. (Bryn)
• 06:21 UT Wind stow released, drives working as normal. (Bryn)
• 06:26 UT Wind stow, missed scans 289-0626 to 289-0632
• 06:45 UT Wind stow, missed scans 289-0648 to 289-0650
• 07:19 UT Wind stow, missed scans 289-0718 to 289-0808
• 07:30 UT Wind stow, but telescope not slewing to stow position - problem with the drives? (Bryn)
• 07:52 restarted fs on Jamie's suggestion, telescope wouldn't release from stow. (Bryn)
• 16:19 UT Wind stow, missed scans 289-1614 to 289-1640 (Ross).
• 17:13 UT Wind stow, missed scans 289-1711 to 289-1718 (Ross).
• 05:22 UT - Clock jump, massive delay differences for 2 scans. Reporting no controller clock. Drives turned themselves off but the wind isn't very high and they didn't try to stow. Halting schedule to reset DBBC and restart drives. Also putting in new module, disk pos 11482 GB. (Ellen)
• 05:50 UT - Won't connect to the mark 5. Jamie is investigating. (Ellen)
• First scan back 290-0604, on new module HOB+0098. (Ellen)
• 07:30 UT - Lost connection to the antenna controller (I think), telescope was stuck and could not be fixed with the standard open or operate commands, needed to use the HMI interface on timehb and do a drive reset and operate toggle to regain connection. At the same time DBBC connection dropped out, had to reconfigure and resync the dbbc to fix it. Possibly another power glitch at Mt Pleasant has caused this? Everything seems fine now and the dynamic scheduler has been restarted. (Tiege)
• 08:30 UT - Everything back up and running, first good scan 290-0834 (Tiege)

Katherine

Disk VSN: HOB+0122

Data volume at beginning = 0 GB

• 23:25 UT enabled dynamic schedule (Bryn)
• 23:46 UT First good scan 288-2347 (Bryn)
• 01:11 UT slewing errors followed by a 2016.289.01:12:18 ALARM: error m5 -900 not while recording or playing
• 01:33 UT same errors as 01:11 UT, seems to be whenever schedule is updated (Bryn)
• ALARM: error m5 -900 not while recording or playing occurring frequently. UT 20:41 ; 22:13 ; 00:45 ; 03:33 (Ellen)
• 00:19 UT - FS crashed while recording, was picked up after 20 minutes from watchdog. Next good scan 290-0057. (Ellen)
• 05:38 UT - Antenna lost connection and no amount of resetting and rebooting has fixed it. Reporting that antenna is stuck. Jamie is investigating. (Ellen)
• First scan back 290-0547. But Rxmon not running until 290-0605. (Ellen)
• 13:10 UT - ALARM. field system crashed after 12:56 UT, last recorded scan 290-1256.
  • Field system restarted and schedule restarted, first scan 290-1322 (Lucas)

Yarragadee

Disk VSN: HOB+0126

Data volume at beginning = 0 GB

• 23:25 UT Yg being taken out of schedule for radioastron in 30 mins (Bryn)
• 03:10 UT Yg finished radioastron, back on dynamic scheduling. First good scan 289-0320(Bryn)