This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
operations:documentation.errors [2017/04/06 01:45] Lucas Hyland [ALARM: Large difference between formatter and maser delays. Check for stability of new offset] |
operations:documentation.errors [2017/08/13 22:15] Tiege McCarthy |
||
---|---|---|---|
Line 74: | Line 74: | ||
First scan of the schedule this came up. The telescope checks the maserdelay and clkoff before every scan and if it is >0.5us then this alarm comes up. I immediately checked the maser first with | First scan of the schedule this came up. The telescope checks the maserdelay and clkoff before every scan and if it is >0.5us then this alarm comes up. I immediately checked the maser first with | ||
- | maserke | + | maserke |
In terminal and it was all green. After that I went to "ke" (pcfske) and checked | In terminal and it was all green. After that I went to "ke" (pcfske) and checked | ||
- | fmset | + | fmset |
And saw there was a large delay between the dbbc and pcfs. I halted the schedule: | And saw there was a large delay between the dbbc and pcfs. I halted the schedule: | ||
- | halt, | + | halt |
- | disk_record=off | + | disk_record=off |
+ | | ||
(in erememote) | (in erememote) | ||
and typed into fmset "s", "y", "s", "y" to resync. After the resync it still wasn't the same so I opened a terminal and typed: | and typed into fmset "s", "y", "s", "y" to resync. After the resync it still wasn't the same so I opened a terminal and typed: | ||
- | vncviewer dbbcke | + | vncviewer dbbcke |
In the dbbc, I closed the 'DBBC Control v104_2.exe' window and reopened the program from the desktop. It asked me to reconfigure and "y". When it was finished I typed | In the dbbc, I closed the 'DBBC Control v104_2.exe' window and reopened the program from the desktop. It asked me to reconfigure and "y". When it was finished I typed | ||
- | dbbc=pps_sync | + | dbbc=pps_sync |
Into eremote and resynced in fmset again. After this it was stable so I restarted the schedule with | Into eremote and resynced in fmset again. After this it was stable so I restarted the schedule with | ||
- | cont | + | cont |
+ | |||
+ | |||
+ | ====ALARM: Field System Time-Out ... etc==== | ||
+ | As it says in the alarm, the first step to check this alarm is to log onto the corresponding pcfs. E.g. | ||
+ | |||
+ | vncviewer pcfske:1 | ||
+ | |||
+ | Or use the convenient VNC pull downs in the menu. Once inside, it should be fairly obvious if the field system is running. However, if you are unsure simply open a new terminal window and type | ||
+ | |||
+ | fs | ||
+ | |||
+ | If the FS is already running, it should return 'field system already running', if not it will start the field system. If the field system IS running, but frozen, close the terminal window it is running in and restart. | ||
+ | |||
+ | Now if the field system is running, but somehow cannot connect to to e-remote control AND/OR the log monitor then this will also cause this ALARM by design. Obviously we want the log errors to be pushed through the pcfs, into the erc and lastly into the log monitor. And if this alarm goes off then this stream has been broken or inactive for too long. | ||
+ | |||
+ | Check the e-remote control. If it is not working then it may need a restart or complete reboot. See if inputs from erc are going into the fs AND the log monitor. A command like | ||
+ | |||
+ | disk_pos | ||
+ | |||
+ | Will return a response from both. If they don't respond after this then the eremote is frozen and requires a reboot. If they do, check the timeout time, if there has been 15mins before the next scan but it is set to 10mins, then this will set off the alarm. | ||
+ | |||
+ | ====ERROR st - 28 UT1TimeOffset non-zero, see value above / ERROR st -27 Computer time difference exceeded 0.25 seconds, see value above (persistent beeping)==== | ||
+ | |||
+ | Most of the notes on the wiki state that this error is benign, however is is very annoying. In order to fix it (FOR NON RADIOASTRON EXPERIMENTS) you must ssh time__ | ||
+ | |||
+ | In the HMI gui, go to the 'SETTINGS' tab. There will be a 'TIME OFFSET' box in the middle. You need to change the 'UT1-UTC' value to 0.0. It will most likely be blank. Save the changes. | ||
+ | |||
+ | Restart the field system and the changes will be implemented. Shout out to Arwin for figuring this one out. | ||
+ | |||
+ | ==== Ceduna Network Down ==== | ||
+ | |||
+ | First ssh into hobart and make sure to forward the X server | ||
+ | <code>ssh -X oper@hobart.phys.utas.edu.au</code> | ||
+ | From terminal on hobart, VNC into the 'manager' PC | ||
+ | <code>vncviewer manager</code> | ||
+ | Go to network connections in the bottom right, left-click on this icon and then right-click on 'CEDUNA-NETWORK' and connect. May need to click 'dial' on a pop up window. Once this connection is established open up TightVNC (should be in start menu). VNC into the router PC, details should already be in, just press connect. **The password is not standard for this PC** (ask for it in group chat and remember/write it down). | ||
+ | |||
+ | Check status of 3G network on the status page, navigate to system tools on the left. Click on the reboot sub-menu. Click reboot and confirm. | ||
+ | |||
+ | Start pinging the ceduna gateway (131.217.61.129) on your PC (not in the VNC window). In a terminal or command prompt: | ||
+ | <code>ping 131.217.61.129</code> | ||
+ | After the router finishes rebooting, click on the status menu (top-left) and check the 3G status. Ideally, the pings will start reaching the the Ceduna gateway, If so skip the next step, If not it may be the VPN that needs fixing, to do this run putty.exe (on the desktop), ssh into 192.168.1.61, login as 'physics' (different password again) and run the following command: | ||
+ | |||
+ | <code>clear crypto ipsec client ezvpn</code> | ||
+ | |||
+ | Once you have established connection (can ping the gateway), exit out of the router PC VNC window, then go down to network connections on 'manager', **right-click on CEDUNA-NETWORK and click disconnect. This is important.** | ||
+ | |||
+ | Now you are done! | ||
+ | |||
+ | Brett is the expert on this setup, so if anything is not as described above, he is the best contact person. | ||
+ | |||
+ | Not really in the ideal spot, however, not sure I have the ability to add to the RA wiki -Tiege 14/08/17 | ||