User Tools

Site Tools


operations:handover

This wiki is not maintained! Do not use this when setting up AuScope experiments!

Handover notes

Please use this page to write up any notes on anything that needs to be passed on to the next observer or should go into the end of experiment message.

Notes should be placed below the horizontal line as a new blog entry.

Continuing problems should be copied to the Current Issues page.

**Template for Handover notes:**

Be sure to add any specific notes about the experiment if required, for example “Special back end configuration for x reason”. Put as much detail as you can - the more the better.

It is ESSENTIAL to put your name after EVERY entry. Copy and paste the template into the new handover notes

====== Experiment Name ======

**Hobart 12m**

Disk VSN: abc-123 | Data volume at beginning: 2.34 GB

   * 1300UT a comment from me (user)

**Katherine 12m**

Disk VSN: | Data volume at beginning:  GB

   * 

**Yarragadee 12m**

Disk VSN: | Data volume at beginning:  GB

   * 1300UT Experiment started okay (JMc).

Format for entering a comment It is important to stick to this formula - there is a good reason we need this information, so please include it!

Time stamp in UT * Comment * (Your Name)

For Example:

20:00UT: Wind stows, Missed scans 012-3456 to 012-3457 (Jim)

20:50UT: Mark5 lost connection, had to restart Mark5 and reconfigure DBBC, called Jamie on call to fix, missed scans 012-3456 to 012-4567 (Jim)

Again - as much detail as you can. It is important to remember that these notes will be sent to the correlator, so please make sure they make sense to others.


AUST68

Hobart 12m:

Disk VSN:HOB+0117/32000

Data volume at beginning: 20786 GB

  • 04:11 UT - Disk_pos 100 GB behind schedule (Ross).
  • 05:51 UT - Disk_pos 130 GB behind schedule (Ross).

Katherine 12m:

Disk VSN:HOB+0107/16000

Data volume at beginning: 6113 GB

Yarragadee 12m:

Disk VSN:HOB+0115/16000

Data volume at beginning: 5699 GB

2014/12/06 23:52 · 0 Comments
AUST67

Hobart 12m:

Disk VSN:HOB+0117/32000

Data volume at beginning: 15.24TB

* S-band: autocorrelation spectra of channels 5, 6 (of 6; i.e. two last) show RFI peaks (Liza)
* Persistent error: "2014.340.04:27:29 WARNING: error s5 -104 rfpcn: time-out, connection closed" (Liza)
* S-band Tsys are elevated (105-175K) (Liza)
* 04:26UTC recorded is appr. 90GB less than in .sum file (Liza) 
* 04:52UTC recorded is appr 100GB less, no obvious loss of data was detected! (Liza)
* 14:00 UT disk pos 300 GB behind sched (Ellen)
* 16:00 UT disk pos 350 GB behind (Ellen)
* 19:00 UT disk pos 400 GB behind (Ellen)
* 21:20 UT disk pos almost 500 GB behind (Ellen)

Katherine 12m:

Disk VSN:HOB+0107/16000

Data volume at beginning: 0TB

* In the set-up phase, the Mk5 time was off and didn't sync with FS time (fmset "s", "=" didn't change it, fmset "." didn't work; restarting mk5ke and pcfske didn't work) It improved by itself at some point (before the start of the experiment)The network was especially slow at that moment.
* S-band: autocorrelation spectra of channels 2, 3, 5, 6 (of 6) show peaks and slopes due to RFI (Liza)
* S-band channels Tsys ~ 140-170K (Liza)
* 15:36 UT: Dimino died for no reason (ALARM: error m5 -900 not while recording or playing), restarted at about 15:30. Scans 340-1535 and 1537 probably bad. (Ellen)
* Some time after this (during scan 340-1537) the field system froze. Everything was still open but I couldn't issue any commands and as far as I could tell, nothing was happening. Had to manually force close it and restart. Probably missed scans 340-1537 to 1604. ~100 GB down (Ellen)
 

Yarragadee 12m:

Disk VSN:HOB+0115/16000

Data volume at beginning: 0TB

  • S-band: autocorrelation spectra of channels 5, 6 show slopes due to RFI (Liza)
  • S-band channels 1, 4, 5, 6 (of 6) have elevated Tsys ~ 105-130K (Liza)
2014/12/05 21:59 · 0 Comments
R1664
Hobart 12m

Disk VSN: NTO-0007

Data volume at beginning: 0

  • Autocorrelation spectra suggest RFI in channels 14, 15 /(0-15): peaks and slope (Liza)
Katherine 12m

Disk VSN: UVLBI-13

Data volume at beginning: 0

  • 1755 UT: Large clock jump. Ran fmset. Seems stable again (DH)
  • 1800 UT: Tsys very high. Has come back down, but still higher than usual (DH)
  • Autocorrelation spectra suggest RFI in channels 11, 12 /(0-15): slope, and hint of slope as well in ch. 5 (Liza)
  • UT 2014.336.14:50:49.01?ERROR st -5 Error return from antenna, see Mbus error – FS crashed and scans lost from this moment onwards.
Yarragadee 12m

Disk VSN: MPI-0153

Data volume at beginning: 0

  • A persistent error: “2014.336.14:40:37 WARNING: error s5 -104 rfpcn: time-out, connection closed” (Liza)
  • Autocorrelation spectra suggest RFI in channels 10-15 /(0-15): small peaks across the bands (Liza)
2014/12/01 17:03 · 0 Comments
AUST66

Observing at Mt Pleasant from 06:00 UT due to internet down at Uni. I can't get any of the log monitors to open log files, so currently the log monitors are not running. (Ellen) The internet is back up so I'm heading back down (09:56 UT) as Katherine is up and running again. (Ellen)

Hobart 12m:

  • Disk VSN: HOB+0117
  • Data volume at beginning: ~10.7 TB

Katherine 12m:

  • Disk VSN: HOB+0122
  • Data volume at beginning: ~10.5 TB
  • 01:48 UT: S-band ch 09 10 are reading as $$$$ but power levels seem OK. (JS)
  • 02:10 UT: Sys temp OK now (JS)
  • 06:10 UT(ish): field system crashed. Can't log into timeke, there's been a power failure so most of the computers aren't running, Martin is driving out there. The generator didn't start on its own, Martin had to do in manually because it was in shut-down mode or something. (Ellen)
  • 07:15 UT: power restored but I can't log into the field system computer. (Ellen)
  • 08:40 UT: power levels all reading 0. Poor Martin is on his way back out there to fix. Mark 5 also playing up (Ellen)
  • 09:30 UT: recording again. Scans missed from around 334-0602 to 334-0930. First scan back was 0930. (Ellen)
  • 10:35 UT: clock offset jump (to about -0.8 us), fixed with counter and fmset. (Ellen)
  • 11:07 UT: same problem (about -0.5 us). (Ellen)
  • 11:40 UT: and again. I'm starting to think there's a problem… (Ellen)
  • 334/12:15 reconfigured dbbc to fix delay drift for last two hours; data for last two hours may be problematic as clock delay drifting in linear fashion, should be good from scan 334-1220 (warren)
  • 334/23:30 rain since around 22:00, and system temperature high at low elevations, S/X 120/110 (warren)

Yarragadee 12m:

  • Disk VSN: HOB+0120
  • Data volume at beginning: ~11.1 TB
  • 09:12 UT: accidentally shut down Dimino (because the shortcut to the mk5 in the menu for Ke is actually for Yg at Mt Pleasant), restarted as aust66ke and then later re-restarted as aust66yg after I figured out what I'd done wrong (about 09:44 UT). So possibly bad scans from 334-0911 to 334-0944. (Ellen)
2014/11/30 00:08 · 0 Comments
AUST65

Hobart 12m:

  • Disk VSN: HOB+0117
  • Data volume at beginning: ~7.73 TB
  • 06:32 recording is 60GB less than planned, at 19:41 already 200TB is missing. No obvious missing scans though…(Liza)
  • Autocorrelation spectra show RFI spikes in ch 14, 15 (count 0-15) and occasional slope in ch 15 (Liza)
  • Persisting error “2014.333.19:45:36.13?ERROR s5 -104 rfpcn: time-out, connection closed” (Liza)
  • Tsys are elevated. In S-band appr 100-130K. In X-band appr 90-100K.

Hobart 26m:

  • Disk VSN: USN-0027
  • Data volume at beginning: ~7.8 TB

Katherine 12m:

  • Disk VSN: HOB+0122
  • Data volume at beginning: ~6.95 TB
  • Autocorrelation spectra: RFI spikes in ch 11, 12, 15, slopes in ch 14, 15 (count 0-15) (Liza)

Yarragadee 12m:

  • Disk VSN: HOB+0120
  • Data volume at beginning: ~7.8 TB
  • autocorrelation spectra show hint of RFI in ch 11 and 12 (forest of small peaks) and clear slope in ch 14, 15 (chnnel count 0-15) (Liza)
  • 150GB of data are missing from recording compare to the sum file, no obvious problems with missed scans at the moment though (Liza)
  • Tsys are fine, with occasional high points in ch 9, 10, 14, 15 (count 0-15) (Liza)
2014/11/29 00:06 · 0 Comments
AUST64

Hobart 12m:

Disk VSN:HOB+0117 (Imogen)

Data volume at beginning:0GB (Imogen)

Katherine 12m:

Disk VSN:HOB+0122 (Imogen)

Data volume at beginning:0GB (Imogen)

  • storm went through from 16:30 looks like lightning strikes close by and around, may have triggered following,(warren)
  • antenna problem at 331/16:46, appears azimuth hard limit reached,(but not in fact) (warren)
  • missed last part of scan 331-1641, missed scans through to 331-1702 (warren)
  • back on track for scan 331-1720b (warren)
  • also clock jump during antenna outage, fixed before recording recommenced, perhaps power glitches (warren)
  • another antenna outage at 331/17:36:37, required stow, fmset, and fs restart (warren)
  • missed last part of scan 331-1731a, through 331-1742
  • back on track for scan 331-1751
  • another antenna outage at 331/17:54:05, required stow, fmset, and fs restart (warren)
  • missed scan 331-1754
  • back on track for scan 331-1805b
  • 331/18:19 clock jump, missed scan 331-1819, while clock reset(warren)
  • rain storms and lightning all around
  • clock offsets drifting steadily apart, reconfigure dbbc at 331/18:40 (warren)
  • missed scan 331-1840, (recording resumed at 18:43:33, but scan likely no good)
  • next good scan 331-1849b (x-band temperature initially high ~ 110) (warren)
  • 331/20:45 data about 245 GB less than scheduled due to missed scans (warren)

Yarragadee 12m:

Disk VSN:HOB+0120 (Imogen)

Data volume at beginning: 0GB (Imogen)

  • High Tsys in chan 13 & 14 (1 indexed) (DH)
  • ~1430 UTC: Several scans late on source (DH)
2014/11/27 02:57 · 0 Comments
/home/www/auscope/opswiki/data/pages/operations/handover.txt · Last modified: 2018/10/29 16:27 by Jesse Swan