AuScope VLBI Operations Wiki

User Tools

Site Tools

You are here: start » operations » handover
Trace: • handover

Sidebar

operations:handover

This wiki is not maintained! Do not use this when setting up AuScope experiments!

Handover notes

Please use this page to write up any notes on anything that needs to be passed on to the next observer or should go into the end of experiment message.

Notes should be placed below the horizontal line as a new blog entry.

Continuing problems should be copied to the Current Issues page.

**Template for Handover notes:**

Be sure to add any specific notes about the experiment if required, for example “Special back end configuration for x reason”. Put as much detail as you can - the more the better.

It is ESSENTIAL to put your name after EVERY entry. Copy and paste the template into the new handover notes 

====== Experiment Name ======

**Hobart 12m**

Disk VSN: abc-123 | Data volume at beginning: 2.34 GB

   * 1300UT a comment from me (user)

**Katherine 12m**

Disk VSN: | Data volume at beginning:  GB

   * 

**Yarragadee 12m**

Disk VSN: | Data volume at beginning:  GB

   * 1300UT Experiment started okay (JMc).

Format for entering a comment It is important to stick to this formula - there is a good reason we need this information, so please include it!

Time stamp in UT * Comment * (Your Name)

For Example:

20:00UT: Wind stows, Missed scans 012-3456 to 012-3457 (Jim)

20:50UT: Mark5 lost connection, had to restart Mark5 and reconfigure DBBC, called Jamie on call to fix, missed scans 012-3456 to 012-4567 (Jim)

Again - as much detail as you can. It is important to remember that these notes will be sent to the correlator, so please make sure they make sense to others.

AOV009

Hobart 12m:

Disk VSN: HOB+0047/6000/1024

Data volume at beginning: 1665.252 GB

  • 18:00 UT: Experiment started OK (Tiege)
  • 05:34 UT: Wind stow started scan 133-0534a is the first affected. (Jim)
  • 05:45 UT: Auto-stow released. Lost all scans until and including 133-0538. Next scan is 133-0554 (Jim)
  • 05:59 UT: Wind stow again. Between scans at the moment. Next one at is 133-0614. (Jim)
  • 06:07 UT: Recovering from stow. No scans lost this time (Jim)
  • 06:41 UT: Another wind stow started (Jim)
  • 06:48 UT: Auto-stow released. Scans affected 133-0641 (stowed mid scan for this) through to 133-0646a. (Arwin)
  • 08:23 UT: disk_pos trails by 56 Gb. (Arwin)
  • Had the following error “WARNING: error st -24 computer time window exceeded 0.25 seconds, see value above” at the times listed below. According to the wiki, it is related to network transfers running at the same time and can be safely ignored.(Arwin)
    1. 07:28 UT
    2. 08:41 UT
    3. 09:08 UT
    4. 12:07 UT
    5. 12:48 UT:
  • 09:41 UT: Automatic wind stow activated. (Arwin)
  • 09:50 UT: Wind stow auto released but antenna did not recover. It went into another wind stow while trying to manually bring the telescope back (Arwin)
  • 10:29 UT: Telescope function restored manually after switching drives off-on. Scans affected 133-0944 through to 133-1027. It might have recorded data for 133-1020,133-1022, and 133-1027 after restarting the schedule. (Arwin)
  • 11:29 UT: ALARM: error m5 -104 mk5cn: time-out, connection closed. Everything seems to be fine, previous handover notes suggest nothing needs to be done. (Arwin)
  • 10:58 UT: Wind stow activated. (Arwin)
  • 11:15 UT: Auto-stow released. Scans affected 133-1113.(Arwin)
  • 11:32 UT: Wind stow activated. (Arwin)
  • 11:56 UT: Antenna back online after wind stow release and manual intervention. Scans affected 133-1135 through to 133-1154. (Arwin)
  • 12:09 UT: disk-pos is lagging by about 105 gb, which is up from 56 gb lag observed 2 hrs earlier. I belive this is due to having the scheduled halted to recover from stows. (Arwin)
  • 12:19 UT: ALARM Automatic wind stow activated. (Arwin)
  • 12:25 UT: Auto-stow released. Scans affected 133-1224 (Arwin)
  • 12:27 UT: ALARM Automatic wind stow activated (Arwin)
  • 12:33 UT: Auto-stow released. Scans affected: None (Arwin)
  • 13:04 UT: Wind Stow activated (Liz)
  • 14:08 UT: Autostow released. Antenna not released. While trying to recover manually, went into another stow (Liz)
  • 14:10 UT: Wind stow activated (liz)
  • 14:27 UT: total data recorded lagging by 110Gbyte, while large, only an increase of 5Gbyte from 2hrs ago despite long stow periods (Liz)
  • 15:25 UT: ALARM: error m5 -104 mk5cn: time-out, connection closed again as listed by Arwin at 11:29 UT, previous handover notes suggest nothing needs to be done (Liz)
  • 16:12 UT: Disk position still lagging, by approx 116Gbyte (Liz)
  • 17:09 UT: Noticed wind speed had dropped and antenna was not stowed in the AneMonitor window but e-remote control and log monitor were still saying stowed. Followed instructions in the wiki for 12m antenna wind stows but once again, just as it seemed fixed, wind speed picked up to 50+ km/h and antenna stowed (approx 17:11 UT). log monitor didn't alarm though (Liz)
  • 17:25 UT: As above (17:09 UT) but while attempting to manually recover started to get error WARNING: error bo -211 can't open a new schedule when an experiment procedure is executing (Liz)
  • 17:44 UT: spoke to Ellen. Unsure why this is occurring. Near end of experiment ? advised that was unlikely to fix, restart the field system and just let schedule end.

Missed scans running tally (for end of experiment email convenience-Arwin)

  1. 133-0534a through to 133-0538 -wind stow
  2. 133-0641 (stowed mid scan for this) through to 133-0646a -wind stow
  3. 133 - 133-0534a through to 133-0538 -wind stow
  4. 133-0641 (stowed mid scan for this) through to 133-0646a -wind stow
  5. 133-0944 through to 133-1027 -wind stow + failure to recover
  6. 133-1113 -wind stow
  7. 133-1135 through to 133-1154 -wind stow + failure to recover
  8. 133-1224 through to 133-1239 -wind stow
  9. 133-1305 through to end
2016/05/11 17:56 · Ross Turner · 0 Comments
r1739

Hobart 12m:

Disk VSN: HOB+0047/6000/1024

Data volume at beginning:22.244GB

  • 17:00UT: Experiment started OK (Jonathan)
  • 00:12UT: Windstowed; missed scans 131-0010 through 131-0015. (Lucia)
2016/05/09 17:02 · Ross Turner
r4738

Hobart 12m:

Neither Arwin, nor I could figure out how to reset the skycam, it is not the same process for Ke as documented on the wiki. (Ellen)

Disk VSN: HOB+0118 (Intended to record on the other module but it reverted to this one during one of the outages/reboot, didnt know this would change the module selected)

Data volume at beginning: 30060.086 GB

  • During the setup had the following issues (Arwin)
    1. 05:03 UT “WARNING: error m5 -900 : not while other activity”. The makr5 had frozen up and needed a reset from the internet power swtich. This was accomplished with Jim Lovel's help. (Arwin)
    2. 05:57 UT mk5=dot? respone was not_synced. Opened FMSET and synced the clocks. This gave back the nominal resposne of syncerr_eq_0. The time at the end was still greater than 5ms, in the range of 18 ms. The time kept drifting to higher and higher numbers and eventually had to get Jamie to help me. Jamie diagnosed it was the ntpq server that wasn't behaving very well (high offset) and he had to reboot the system. (Arwin)
    3. 06:58 UT The Lat_offsets were between 0.02 and 0.04. Zero'd the offsets after running fivept check, re-ran the fivept check to confirm Lat_offsets were below 0.02. (Arwin)
    4. 11:59 UT WARNING: error st -27 computer acu time difference exceeded 0.25 seconds, see value above. Wiki suggested ignoring this error as it most likely is resulting from an ongoing etransfer?
    5. 17:02 UT Had the following error a few times: WARNING: error st -5 error return from antenna, see mbus error, ''antenna=open' fixed this. Brett emailed regarding a power failer at mt Pleasant around this time.
  • 18:30 UT Experiment started ok. (Arwin)
  • 18:35 UT Onsource stattus is slewing error for scan 126-1835. (Arwin)
  • Had the following wind stows (Arwin)
    1. 20:12:34 - 20:18:17 UT Scans affected 126-2012a through to 126-2014
    2. 21:23:14 - 21:32:11 UT Scans affected 126-2123 through to 126-2130b
    3. A few status is slewing errors while the telescope was stowed.
    4. 21:32:19 - 21:46:08 UT Scans affected 126-2135b and possibly 126-2142 as there was a status is slewing error
  • On source status is slewing errors
    1. 00:35 UT, scan 127-0035 (Arwin)
    2. 04:52 UT, scan 127-0452a (Arwin)
    3. A few while the telescope was stowed due to the wind. Check the times under wind stows. (Arwin)
    4. 06:40:50 UT, scan 127-0640b (Arwin)
    5. 06:42:36 UT, scan 127-0642a (Arwin)
2016/05/05 06:16 · Ross Turner · 0 Comments
r1738

Hobart 12m:

Disk VSN: HOB+0118 Data volume at beginning: 27003.557 GB

  • 17:00 UT - Experiment Started OK. (Ross)
  • 17:00 UT - Experiment Ended OK. Total data recorded 3048.581 GB (Lucas).
2016/05/03 13:36 · Ross Turner · 0 Comments
R4737

Hobart 12m:

Disk VSN: HOB+0118 Data volume at beginning: 25571.0 GB

  • 18:30 UT - Experiment Started OK. (Tiege)
  • 10:50 UT - onsource status is slewing between source 0743+277 to 0131-522 (scan 120-1122 to 120-1129) (Jonny)
2016/04/28 18:35 · Nick Ham · 0 Comments
r1737

Hobart 12m:

Disk VSN: HOB+0118

Data volume at beginning: 24161.032 GB

  • 07:00 UT during setup, both pcfshb and mk5hb hung up when rebooting, though Jamie was able to fix both onsite. (NH)
  • 17:00 UT Experiment otherwise seems to have started Okay. (NH)
  • 17:30 UT Tsys values are all ?????? or in the thousands. (NH)
  • 19:15 UT Getting quite a few of “ERROR s1 -104 ifpcn: time-out, connection closed”, consequently each scan was starting late and missing data. Restarting Rxmon did not fix the problem. (NH)
  • 22:30 UT Jamie fixed the ifpcn errors by fixing up monica. First full scan is 117-2239.
  • 23:00 UT data recording is now 50 GB behind. (Lucia)
  • 23:45 UT Generator state in the System monitor was 'in auto off', but orange. Called Warren at Mt Pleasant, the generator is ok. Restarting the System monitor fixed it. (Lucia)
  • 04:09 UT got a couple of ACU-Computer time errors. All good again after a few minutes (Lucia)
2016/04/26 17:10 · Nick Ham · 0 Comments

<< Newer entries | Older entries >>

/home/www/auscope/opswiki/data/pages/operations/handover.txt · Last modified: 2018/10/29 16:27 by Jesse Swan

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki