• Background reading for new operators
  • About the AuScope VLBI Project
  • VLBI: A fascinating technique for geodesy and astrometry by Schuh and Behrend (2012)
  • International VLBI Service for Geodesy and Astrometry by Nothnagel, Arts Behrend & Malkin, 2016
  • The AuScope Geodetic VLBI Array by Lovell et al.
  • Operator training documentation (PDF). Includes project background, info on scheduling and rosters, overview of the telescopes and related hardware.
  • Acronym dictionary
  • Observer duties and shift times
• More background reading
  • There's lots of useful information in the 7th Technical Operations Workshop Notebook. A couple of items of particular interest:
    • Science Applications of Geodetic VLBI by John Gipson (2013)
    • Pointing and Single Dish Amplitude Calibration Theory by Uwe Bach
  • Maps

Off-site experiment monitoring notes

Below are the links to a step-by-step procedure for making IVS observations on the 12m and 26mtelescopes. The current version of e-RemoteCtrl has a built-in checklist which can now be used during the setup procedure. So the first task is to start the Observing and monitoring software at each site, then e-RemoteCtrl.

• Observations with 12-m telescopes
  • Setup using scripts
  • 12-m checklist parameter description

• Observations with 26-m telescope
  • 26-m checklist parameter description

• New procedures for the Mixed-Mode (AUM) experiments with Hobart12 (Hb) and Katherine (Ke) telescopes

Parkes monitoring and set-up instructions

These are notes on IVS observations from ops8, the six-monitor PC in the AuScope VLBI Operations room in Hobart.

If you are unfamiliar with the computers and software used at the sites and in the control room, please read this before continuing:

• Introductory notes on computers and software

• Preparing the schedule and procedure files
  • Open a terminal and ssh into observer@ops-serv2 and for each station run:

    ~/slogit.auscope.sh

  • Sometimes ~/ is not needed before the slogit command.

* Note - this script will now ask if there is a SAR satellite pass to be observed during the experiment (or immediately afterwards). Please check the Google calendar before answering! If there is a track, then the stations SNP file will be automatically edited to accommodate the scan & you will be given a list of shortened or skipped scans - record these in the handover notes!

• During normal telescope operation you can do all telescopes with one command via

  schedules <experiment name> <st1> <st2> ... <st4> 

  or

  schedules $(nobs -p)

  after checking that this will produce the correct result by typing

  nobs -p

  into the terminal. Note that this will assume that there are no SAR tracks during the schedule to be drudged.

• On occasions, the Hobart data are written to disk between scans for e-transfer to Tsukuba for ultra-rapid EOP observations. Some additional changes to the schedule files are required in this case:
• Additional schedule preparation for Hb ultra-rapid EOP experiments
• The status of the Ultra-rapid experiments is published in real-time on the Tsukuba Ultra-rapid web page.

Check that the software on the timing PCs is running:

• Software on the timing PC

• Starting the PC Field System Software
• Some notes on antenna control and monitoring using the new version of the Field system

• To quickly open the necessary windows on ops8 for monitoring the telescopes (e-RemoteCtrl, Log Monitor, System Monitor, delaylive, tsys monit3) first make sure these are already closed for each telescope you plan to open them for by entering

  windex

  into a terminal (and manually closing any stragglers). Then enter

  windows hb ke yg

  feel free to omit any telescopes not in use.

• Following this enter

  windowsweb

  to open browser windows to the VLBI webcams, Autocorrelation spectra, and Auscope Handover page.

• You will need to login to each e-RemoteCtrl window, start recording the log, and open that log in the Log Monitor as instructed below.

• Start the System Monitor

• Start MONICA

Next the Mark5 recorder needs to be prepared for the observations.

• Mark5B recorder setup

• Start e-Remote Ctrl
• Open a new PCFS log file
• Next, set up the log monitor to provide warnings and alarms
• Now start the Checklist by loading the procedure file for the experiment, then exper_initi. e.g for R1556 at Hobart:

  proc=r1556hb
  exper_initi

The checklist should now be visible. The items in the list are described below and can be ticked off as you go

• For your convenience you may wish to use Multi Command Input – moprin (written by Dave). The link to moprin is located in the upper left screen (alternatively open a terminal on ops8 and type 'moprin'), together with the other helpful tools like Tsys displays and backup control – ssh-based remote operator input. With moprin you can send the same command to all three telescop operator input simultaneously. Useful for pointing, ckloff, maserdelay, terminate at the end of an experiment and so on. Also you can incert the telescope name by typing $s instead. For example, if you wish to set up relevant log names at each station, you can type

  log=crf86$s

  You can use it for setting up procedures and starting schedules as well. For more information (e.g. how to change the stations, type 'help' into the moprin terminal.

The RF and IF signal paths, and DBBC are now configured through the Field System.

• Configuring the RF and IF signal paths
• if Hb reports 12,0,10,10 and you want 0,0,10,10 like it says on the above page, type “ifpic=,,0,0,10,10,” into eremote (Lucas).

The Digital Base-Band Converter (DBBC) takes the IF signals from the telescope and digitises them for the Mark5 recorder. The DBBC needs to be properly configured prior to an observation. Check that the DBBC is healthy and the server software is running:

• DBBC Setup

To configure the DBBC with the correct IF and frequency settings, use the following command

setupsx

This could be setup8f for the 512 R1, or setup01 for Hart schedules 'crds—', some 'aua—' setupsx calls ifdsx which sets up the DBBC Conditioning module inputs, filters and power levels, then dbbcsx8 (or dbbcsx4 for a 4 MHz DBBC mode) which sets the BBC frequencies. Then if you type:

iread

You will see what the Conditioning module settings are. Output format is:

<time>/<Module label>/<IF input number>,<Auto or Manual gain control>,<Nyquist filter number>,<Target power level>,<attenuation>,<actual power level>

Check there’s agreement with what appears in ifdsx, as well as the values listed here. The actual power level should agree pretty well with the target level. The attenuation number can be anywhere between 0 (none) and 63 (maximum). If you see it at 0 or 63, it means the Conditioning module is having trouble getting the power to the right level. See the Current Issues page for information on how to do this.

Then type:

bread

You will see what the DBBC has set the BBC freqs to (compare with the .prc file). Output looks like this:

<time>/<bbc name>/<Frequency (MHz)>/<Conditioning module in use>,<Bandwidth (MHz)>,...

the Frequency, Conditioning module label and bandwidth should agree with the listing in dbbcsx8 (or dbbcsx4).

• Hydrogen Maser check

Check that a suitable module is in the Mark 5 recorder.

• Which module do I use?
• Module and Experiment Library

If there is no specific module allocated to the experiment, the relevant column on the spreadsheet will be left blank.

• Checking clock offsets and delays

• Making a test recording

• Tsys measurement check
• Note that Ke is now using a different continuous Tsys measuring procedure. These notes on Continuous calibration explain what's going on.

• Antenna checks

• Weather conditions

• Fringe checking procedure

• Start Skype and begin a chat session with all participating sites plus the on-call person

Lastly, after submitting the checklist form, send a ready message to IVS:

• Sending a station ready email to IVS

• Starting and monitoring the observations

At the end of an observation:

• Finishing up
• Following on with another experiment
• Processing logs and shipping data

• eVLBI

• Schedule file preparation
• Observing
• Correlation

New content! Please add! Here is an ever growing list of common errors and alarms that you may come across whilst observing. Some are diagnostics and others are plain alarms. If you come across more while observing please add them to this page and note the data and time of the occurrence and your fix. Or, if you couldn't resolve it leave it blank so observers know that it is a problem that was encountered and this would be a good time to call an on-call person List of alarms and fixes.

1. Before you drudg, start the correct fs-mk5hb (fs-auscope?) for IVS which runs a script to select the correct backend (mk5 and DBBC). Drudg schedule for the experiment (if it has not been done already). Drudg the schedule using slogit.auscope.sh on ops-serv2 as for the other stations. Having the backend selected correctly will result in the correct proc file being generated.

From ops8, confirm that the sum file has been copied across to /vlbobs/ivs/sched/. Otherwise, ssh into hobart (ssh oper@hobart) and copy the .sum file with the command

scp /tmp/sched.tmp observer@ops-serv2:/vlbobs/ivs/sched/aug019ho.sum

.

If the .sum file was not written:

a) before the experiment started: drudge the schedule manually. On Hobart, go to the sched directory and type drudg r4705.skd. Then type Ho for station Hobart and then 3 to create the prc file and 5 to write the sum file. The sum file is written in the ./tmp directory as sched.tmp.

b) after the experiment has started: in order not to overwrite the prc file, copy the schedule file to the tmp directory and drudg it there.

2. Check to see if Jim P (or anyone else!) is observing.

CHECK THE CALENDAR!

Go to the 26m Live Page for the 26m and see if the telescope is tracking. Jim P puts a sticky note on the screen 4 of the newsmerd:1 VNC session where the Field System is saying he is observing also.

Please wait until the other person's experiment has finished until you set up.

If it is urgent, please see the notes on his instructions page on how to cancel the Vela observations:

http://ra-wiki.phys.utas.edu.au/index.php?n=MtPleasant26m.PulsarObserving

Step 1. Open a VNC viewer to newsmerd

vncviewer newsmerd:1

The password is the usual one. (hobart26 FS runs on computer named hobart). If this is not accessible ssh to observer@newsmerd then restart the vnc with

 vncserver :1 -shared & 

Step 2. Check that the field system is running on newsmerd. Usually, the Geodesy window will be Desktop 4. If it is not running, please start it with fs-mk5hb to ensure that the geodetic backend is selected.

Step 3. Check receiver position

Check the receiver position with the command rxp. It should be -256, +248, +18 (x,y,z).

Be sure the telescope is stowed BEFORE changing the reciever

To change it go to Desktop 3, bottom right window and enter '1' (OTTER interface). If the OTTER interface isn't open, then open a new terminal and type ./OTTER_interface.sh. Changing the receiver takes a couple of minutes (check on the live page http://ra-state.phys.utas.edu.au/ until the antenna stops moving - 5 sec update rate).

Check that the receiver is actually moving, and wait until it reaches position before moving the telescope (can take a few minutes if moving from L-Band).

Stuck Receiver

If the receiver is stuck, first try a reset in the OTTER interface on the newsmerd vnc desktop 3 (type r and hit enter).

Sometimes the OTTER interface crashes/fails to communicate with the reciever. If the GUI fails to start or returns error messages, please check that it is running (with ps -ef | grep OTTER as observer@rx26m). You can restart the server with sudo /etc/init.d/OTTER restart if there are any problems. The server is being continuous polled by MoniCA with the data being logged in the hobart database. More information on the OTTER server can be found hereOTTER

Step 4. Check the procedure file The procedure file will be in /usr2/proc and called <experiment name>ho.prc. It should not need editing, and should be similar to recent .prc files using the same mode\

Step 5. Mark 5 setup:

The 26m now uses uses mk5hb (oper@mk5hb) for recording.

Log in to oper@mk5hb and confirm that DIMino is running.

ps auxww | grep -i DIM

If DIMino is running, it will return an output like this:

root     10832  5.2  0.5  40564 12984 ?        Rl   May08 2666:16 DIMino -m0
oper     32071  0.0  0.0   2852   544 pts/2    D+   04:14   0:00 grep -i DIM

If the first line in this output does not appear, DIMino is not running. Run it with

DIMino -m0 &

Check the VSN of the module using

tstDIMino
vsn?

Check if there is any data on the module by using the command DirList. If there is (and there is supposed to be data on there), note the end pointer (as in how much data has been recorded so far on that module) in the Handover Notes

NOTE: If there is no free space on a module in the Mark 5, call your on-call person for assistance in changing modules.

Step 6. Start eRemoteCtrl.

It is availalbe from the drop down menus as it is for the 12m telescopes.

Step 7. Start the Log Monitor as for the 12m telescopes (from the drop down menu). Start a new log file and begin recording to it.

Step 8. Load the experiment procedure file (if you haven't already)

proc=rd1234ho

note the 'ho' on the end for Hobart 26m.

Step 9. Run the setup of the experiment using

 setupsx

1. Run the setup. The command for an R1 experiment is setupsx but look in the experiment procedure library printout to find the correct setup procedure name. Watch for warnings.
2. Check all the values reported by iread and bread are nominal.
3. Check times are ok with fmset on oper@hobart.
4. Set the cal to be controlled by PCFS.

   observer@newsmerd: calu -m pcfs -r rakbus

5. Check that caltsys works. Caltsys should return results between 50 and 70K
6. Check that weather works. wth
7. Check that fmout-gps works. clkoff. Note that the readings are collected from pcfshb at this stage. If clkoff returns an error, or fixed readings, check that the FS on pcfshb is ruuning & restart it if necessary.
8. Check that the Mark5 has a blank module big enough for the experiment. Compare the Drudg summary of the schedule with the label on the front of the disk module.
9. Run disk_pos to check the disk module is empty. Result should be 0/0 otherwise the disk module contains data.
10. Record some data. disk_record=on
11. Observe green disk activity lights on module
12. Stop recording. disk_record=off
13. Check the recording. with checkmk5
14. Observe the time and details in the log window for the test recording, and have a look at the autocorrelation spectrum and bit statistics screens.
15. Start the schedule. schedule=r1186ho,#1
16. Watch PCFS log for errors.
17. Check the antenna is on source. onsource

• open the monica client to monitor the He pressure

Lastly, send the start message.

Once the schedule starts, watch the log and everything closely for the first scan or so.

Sometimes the drives need to be manually re-started. Documentation on how to do this can be found here.

• Please note that e-RemoteControl is now longer supported and will not connect to the field system!
• Instead, please use stream_log ho to create a local file for the log monitor.
• Checklists now have to be manually triggered with checklistHo $exper (where $exper is the experiment name) - to launch the start or end checklists, please use checklistHo $exper start or checklistHo $exper stop repectively.
• Parameters of the check-list during observation for 26-m are explained here.

Send the end message

Park the dish with

 source=stow 

in the Field System

Setting up and monitoring Parkes instructions.

Gmail accounts have been set up for registration with skype and other services. The usual Observer password applies.

gmail: keobserver@gmail.com

Skype: keobserver

gmail: ygobserver@gmail.com

skype: ygobserver

gmail: hbobserver@gmail.com

skype: hbobserver