User Tools
operations:agenda2014_12_18
This wiki is not maintained! Do not use this when setting up AuScope experiments!
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
operations:agenda2014_12_18 [2014/12/17 23:02] liza |
operations:agenda2014_12_18 [2014/12/18 02:26] (current) liza |
||
|---|---|---|---|
| Line 5: | Line 5: | ||
| 1. Current issues: | 1. Current issues: | ||
| * Observing at Mt Pleasant: experience, suggestions. | * Observing at Mt Pleasant: experience, suggestions. | ||
| - | * Hb recording consistently less than schedule assumes for certain experiments. | + | * Alarms at Mt Pleasant -- what is wrong? (seemingly nothing) |
| - | * Ke time setting -- doesn't synch | + | * alarm is .ogg maybe not playing -- convert to another format? |
| - | * Alarms at Mt Pleasant -- what is wrong? (seems nothing) | + | * another another player? |
| + | * use the system alarm at Mt Pl - mute! | ||
| + | * Hb recording consistently less than schedule assumes for certain AUST experiments (e.g., [[handover:aust67|AUST67]], [[handover:aust68|AUST68]]) | ||
| + | * Check scan_checks (Jamie) | ||
| + | * Ke time setting -- mark5 doesn't synch (Recent [[handover:aust72|AUST72]]+generator issue) | ||
| + | * Ke issues during [[handover:aust67|AUST67]] and [[handover:crds74|CRDS74]] //(see below)// | ||
| + | * MONICA puts load on pcfs, but maybe not it is to plame | ||
| + | * java? | ||
| + | * keep monitoring and gather info | ||
| + | * Ed's suggestion: add handover comments to the log rather than to the end message. | ||
| 2. Roster, any questions or comments | 2. Roster, any questions or comments | ||
| Line 13: | Line 23: | ||
| 4. Other items | 4. Other items | ||
| - | * Drinks on Monday jee! | + | * AuScope Observers Holiday drinks 5:30pm Monday 22nd of December in The Winston http://www.thewinstonbar.com/ |
| + | |||
| + | |||
| + | PD Below an email from Ed about recent Katherine issues: | ||
| + | |||
| + | <code> | ||
| + | Hi Jim, Jamie, Brett, | ||
| + | |||
| + | Rich, Dan, and I have been looking at some recent problems Katherine | ||
| + | had. Below is our diagnosis of what happened and some suggestions for | ||
| + | handling them. Please share this with the operators if you think it | ||
| + | would be helpful. | ||
| + | |||
| + | AUST67: The major problem was not a Mark 5 problem, but it was the | ||
| + | tip-off "canary in the coal mine". The FS PC appears to have been loaded | ||
| + | (as happened in CONT) slowing the system down. This snowballed into the | ||
| + | other errors. What was visible was the Mark 5 rejecting commands ("error | ||
| + | m5 -900 not while recording or playing"). When problems like this are | ||
| + | encountered, we recommend that the operator check the load with "uptime" | ||
| + | and enter the numbers in a log comment. Until this issue is resolved, | ||
| + | you might consider having a window opening running tload, xload or top, | ||
| + | as part of normal operations for a visual indication of the load. A FS | ||
| + | PC reboot may fix this problem when it occurs. Ideally, the problem | ||
| + | program should be identified and fixed or at least the operator given | ||
| + | instructions on what program to "kill", which would be less traumatic | ||
| + | than a FS PC reboot. | ||
| + | |||
| + | CRDS74: Again the Mark 5 was not the issue. The DBBC had 1 PPS problems, | ||
| + | visible as a jump in /dbbc/pps_delay/ output. This snowballed into the | ||
| + | other problems. The situation was visible as a Mark 5 time error | ||
| + | ("ERROR sc -13 setcl: formatter to FS time difference 0.5 seconds or | ||
| + | greater"). We suggest that you have the operator monitor | ||
| + | /dbbc/pps_delay/ value and look for jumps. The DBBC needs to be | ||
| + | resync'd/rebooted in this situation, but not the FS, or Mark 5, as the | ||
| + | operator discovered. There is a learning curve. Of course, after | ||
| + | resyncing/reboot the DBBC, the Mark 5 has to be resync'd and the time | ||
| + | set. It seems like there are still DBBC PPS stability issues that need | ||
| + | to be addressed. | ||
| + | |||
| + | BTW, we noticed the following text in crds74ke end message: | ||
| + | |||
| + | Comments from the log: | ||
| + | Disk VSN: WSRT-049 | ||
| + | Data volume at beginning: 1.159 GB | ||
| + | UT 17:10 Sched start. First scan 343-1719 (-13.4 GB) (JS) | ||
| + | UT 10:36 - Mark5B drifting wildly. Halted schedule. (vk) | ||
| + | UT 10:54 To fix clock drift, restarted field system, GPIB and Mark 5 with no luck, reset counters and fmset with no luck, had to restart DBBC, finally fixed the problem. (Imogen) | ||
| + | UT 11:50 - Schedule resumed. Missed scans: 344-1027 to 344-1141, corresponding to ~90 GB (vk) | ||
| + | |||
| + | These comments don't seem to actually appear in the log. Did we just not | ||
| + | look in the right place? Maybe they are just the operators hand written | ||
| + | notes entered for the end message. If so, it would be very helpful if | ||
| + | they could be injected into the log file (a la the 'msg' program) since | ||
| + | these comments contain valuable information that didn't otherwise appear. | ||
| + | |||
| + | Thanks, Ed | ||
| + | </code> | ||
/home/www/auscope/opswiki/data/attic/operations/agenda2014_12_18.1418857370.txt.gz · Last modified: 2014/12/17 23:02 by liza
Page Tools
