====== AUSTRAL 15-day Observations : Nov 28 to Dec 16 2013 ====== * [[#schedule_overview_and_module_usage|Observing schedule and module usage overview]] * [[#schedule_changeover|Schedule changeover procedure]] * [[#module_swaps|Swapping Mark5 modules]] * [[#observatory_checking|Observatory checking procedure]] (every 3 days) * [[#fringe_checks|Fringe Checks]] ===== Schedule Overview and Module Usage ===== Please read the [[operations:documentation_austral15day_overview|Overview Page]] for information on the observations, start and stop times etc. The google calendar on the [[operations:schedule|schedules page]] describes experiment and shift times. The AUSTRAL schedules are named ''A13nn'' where ''(nn = 01, 02, 03, ... , 15)''. Interleaved with these are regular IVS sessions R1614, CRF79 and R4615. There is only a 5 minute gap between AUSTRAL observations, just enough time to change schedule files and modules (if needed). Every three days, an hour is set aside for detailed checks at each site. They are staggered so that only one antenna is out of the array at a time. Check times for AuScope sites are as follows: ^ Site ^ Check time (UT) ^ Check time (Hobart) ^ | Hb | 01.00 - 02.00 | 12.00 - 13.00 | | Ke | 02.00 - 03.00 | 13.00 - 14.00 | | Yg | 04.00 - 05.00 | 15.00 - 16.00 | We are using 16 TB modules at each AuScope station. They will hold two days of data each, so module changes will occur at every second schedule change. The schedule files are written such that the same source is observed at the same sidereal time every day, so the actual schedule changes are NOT at the same UT every time. Note also that R1614, CRF79 and R4615 will go on different modules to the AUSTRALs. Please refer to the [[operations:start#auscope_ivs_module_allocation|Module Usage Spreadsheet]] for a list of which modules to use for which experiments. The sections below describe how operations differ from normal. Note that it will be possible to monitor the Hart 15m antenna and possibly Warkworth as well. ===== Schedule files ===== All schedule files for the period from A1301 to A1315 should already be prepared and ready to use before your shift. However, if you are starting R1614, CRF79 or R4615, you will need to make the usual schedule file preparations. ===== Monitoring observations ===== Please carry out the regular two-hourly checks as normal during the observations. * [[operations:documentation.eRemoteControl_checklist_midob|Description of checklist parameters]] for AuScope telescopes It is also possible to monitor the 15m telescope at Hartebeeshoek and you may be asked to respond to problems when local staff are unavailable. * [[operations:documentation.monitor_hart15|Monitoring the Hart 15m telescope]] It may also be possible to monitor the Warkworth 12m antenna. Details to follow if this becomes possible. ===== Schedule Changeover ===== There is only a 5 minute gap between the AUSTRAL schedules. When no module swap is required, the last line of each schedule will be a command to start the next one (unless the next one is not an AUSTRAL experiment) so it should happen automatically. However, if a module change is required, the changeover should be done by hand. Here's what to do, and as an example, here is the swap from A1302 to A1303 at Hobart. All commands can go in eRemoteCtrl: - At least 10 minutes before the schedule ends, bring up the control room camera display and read the module VSN numbers in both banks. Confirm that the module you are going to swap to is there and is currently inactive. The control room Pan-Tilt-Zoom (PTZ) web cameras are: * Hobart: http://ptzhb.phys.utas.edu.au * Katherine : http://ptzke.phys.utas.edu.au * Yarragadee : http://ptzyg.phys.utas.edu.au \\ These cameras have pre-defined way-points (menu in lower right). Select the Mark5 recorder (e.g. for Hobart it's "mk5hb"). You may have to pan and zoom a bit to read the label. - Wait for the schedule to finish - Change to the station log: log=station - Change modules mk5=bank_set=inc You should see on the camera that the red activity light changes from one module to the next. - Check that the VSN you read off the camera display agrees with the one shown by the Field System ready_disk mk5=vsn? If it doesn't, then the VSN will need setting: - [[operations:van_setting|Setting a module VSN code]] - Now start the new schedule. Include the '',#1'' suffix so that you start the schedule on the first line, e.g. schedule=a1303hb,#1 Once the new schedule has started: * [[operations:startmessage|Send a start message to IVS]]. * Send off the log files to IVS. Refer to the page [[operations:documentation.ivs.commentsfile|Prepare the comments file and transfer the logs]] for details. Note that all modules for AUSTRAL experiments from Yarragadee and Katherine are shipped to Hobart, and all AUSTRAL data are eTransfer-ed to Curtin for correlation. ===== Module Swaps ===== Module swaps should be coordinated with local staff at the observatories. Someone at the site will phone when they want to change a module. Estimated module change times are available here: * [[operations:documentation_austral15day_module_swaps|Module swapping timetable]] It's important that the new module is powered down and up during a scan gap as this draws significant power and the Mark5 can't record to the other module at the same time (and will probably crash). So, when someone from the observatory calls, consult the observation summary file and find an upcoming period of time of at least one minute between a ''Stop Data'' and a ''Start Data'' to carry out the swap. For example, in the schedule below, the period between Scans ''332-2104'' and ''332-2110'' would be a good choice as there's a gap of almost 4 minutes (from ''21:06:18'' to ''21:10:01''). However, a module change between Scans ''332-2118b'' and ''332-2122'' would not be a good idea as the gap is only 25 seconds (from ''21:22:29'' to ''21:22:54''): Start Stop Record Scan Line# Source Az El Cable Data Data Dur Gbyte date = 2013NOV28 DOY = 332 332-2101 7 0648-165 254 6 CCW 21:01:11 21:02:05 0:54 .0 XXX 332-2102b 24 1606+106 50 27 NEUTR 21:02:57 21:04:13 1:16 6.9 * 332-2104 41 1622-253 89 48 NEUTR 21:04:52 21:06:18 1:26 16.6 * 332-2110 58 0454-810 188 31 CW 21:10:01 21:11:30 1:29 27.6 * 332-2111 75 0230-790 184 26 CW 21:11:54 21:18:16 6:22 39.0 * 332-2118b 92 2052-474 140 13 CW 21:18:58 21:22:29 3:31 87.9 * 332-2122 109 2300-683 166 18 CW 21:22:54 21:24:35 1:41 114.9 * 332-2125 126 0727-115 260 6 CW 21:25:51 21:26:21 0:30 127.9 * === Procedure === * Wait for the scan to finish and observe that the four disk activity lights stop flashing * halt the schedule. This makes sure that no recording can possibly happen during the swap and crash the Mark5: halt * Ask the local person to turn the key on the used module and swap it with a blank one. * Wait for three green lights above the new module to indicate it is powered up and ready. * Resume the schedule with the cont command cont * Make a note in the log that the modules have been swapped. Include VSN codes in the message. * Send an email to auscope-operations@list.utas.edu.au so that we know what's happened ===== Observatory Checking ===== Note that the observatory checking procedure requires you to interrupt the schedule and miss some scans. This allows us to skip checks if necessary but also means if the checks take less than the allocated hour we can be observing again as soon as possible. Firstly, check the schedule summary printout and find out which is the first scan to finish after the start of the check period. The idea is to stop the schedule while no data are being recorded. For example, if the check time is 19:00 UT, then in the schedule below, it could be halted after 19:05:45 UT but before 19:07:43 UT: Start Start Stop Record Scan Line# Source Az El Cable Record Data Data Dur Gbyte date = 2013NOV21 DOY = 325 . . . 325-1854 140 1610-771 170 6 CCW 18:53:58 18:54:08 18:58:33 4:25 35.0 * 325-1859 159 1255-316 119 19 CCW 18:59:02 18:59:12 19:05:45 6:33 43.8 * 325-1907a 178 0131-522 218 7 CCW 19:07:43 19:07:53 19:20:57 13:04 56.7 * . . . Now, wait for the scan gap, and then type: halt Now make a note in the log that the schedule is being stopped for system checks. e.g. "Halting schedule for system checks We want to keep all the check data in a separate log, so open a log file called "check.log": log=check At this point it's probably worth verifying that the recorder has stopped: mk5=dot? If the response includes the text ''FHG_off'' then the recorder is off. However, if you see ''FHG_on'' then you're still recording, so stop the recorder with disk_record=off If not running already, [[operations:documentation:start_monica|Start MONICA]] and choose the pre-set ''aust_check'' profile from the GUI for the antenna. e.g. for hobart Navigator -> hb -> aust_check This will plot antenna coordinates, tracking errors and motor currents against time. Now the checks can begin: - Send the antenna to (Az, El) = (1, 87) deg test1 - Wait for it to get there (look for ''flagr/antenna,acquired'' or watch the system monitor display). While you're waiting, Check the generator status: - At Hobart and Katherine you can check the status of the backup generator via the System Monitor interface. The default mode is "In Auto, off" which means it's ready to turn on if there's a power failure but it's currently switched off. If it's in any other state, query it with the On Call person. - Check that the RF and IF signal paths are correct and the attenuators are at their nominal levels. Refer to the notes on [[operations:rf_if_check_9.10.5|configuring the RF and IF signal paths]] - Check the DBBC is properly configured: iread You will see what the current Conditioning module settings are. Output format is: Check there’s agreement with what appears in the ''ifdsx'' definition in the procedure file. The actual power level should agree pretty well with the target level. The attenuation number can be anywhere between 0 (none) and 63 (maximum). If you see it at 0 or 63, it means the Conditioning module is having trouble getting the power to the right level. You may want to adjust the attenuators (see [[operations:rf_if_check_9.10.5|configuring the RF and IF signal paths]]) to get them back in range. - Then type: bread You will see what the DBBC has set the BBC freqs to (compare with the .prc file). Output looks like this: the Frequency, Conditioning module label and bandwidth should agree with the listing in ''dbbcsxd''. - Check the maser. See the[[operations:maser_check|Hydrogen Maser check]] notes - check that the clock offsets are in range clkoff maserdelay and check the difference reported in the log monitor - Make sure Skype is running to allow a chat session with all participating sites plus the on-call person - Use the "Audio test" button on the log monitor to check the alarms are audible. - Measure the Tsys while at a high elevation systemp12 - [[operations:documentation.test_antenna#antenna_pointing_check|Check the pointing]] - There are a series of Az/El slews to move the antenna over it's full range. Type each of these commands into eRemoteCtrl in the order they appear. //Wait for eRemoteCtrl to report ''flagr/antenna,acquired'' before continuing to the next//: test1 test2 test3 test4 test5 test1 These commands send the antenna to the following set of (Az, El) coordinates in turn : (1, 87), (1, 6), (1, 45), (-269, 45), (269, 45), (1,87). The idea here is to exercise the antenna and log the results se we can look for any changes over the days that may indicate mechanical problems. - Lastly, export the data collected by MONICA from the MONICA GUI as follows: - Select "Export -> ASCII Data" - When asked for a file name, give the current obs code and station name (e.g. a1308hb), the word "check", and end with ".txt". For example: a1308hb_check.txt - Select "Export -> PNG image" - As above but a ".png" extension. e.g. a1308hb_check.png Now you can re-start the schedule. Just use the schedule name without any line number suffix. e.g. schedule=c1406hb setup8f This will start the next scan that occurs at least 5 min in the future. Check that the antenna goes to the next source in the schedule that satisfies this criterion. Now make a note in the log that the schedule has been restarted following system checks. e.g. "Schedule resumed following system checks Also, if there were any problems during the checks or anything else of note, now is a good time to do it. For example, if the weather was too cloudy to verify the pointing: "Pointing check not successful, probably due to bad weather ===== Fringe Checks ===== On 1921-293 every day. Procedure?