SUSI Troubleshoot'n

If a clock display appears, flashing, at the top of a UI 
 AND you can get a '68K monitor' prompt hit '?' to get a menu. 
 Hit 'Y' to toggle the 'Time display' off.
This doesn't work for sidcons, for complicated reasons I can't get 
my head around just yet.

Problem: Siderostat reaches a limit
message from sidcon like "...reached pre-lim" or "...pre-lim-lim"

There is no limit for altitude, the sids just keep rotating
But in azimuth there are limits near true west:
  at 250deg going ccw (but avoid going past 260)
  at 290deg going cw  (but avoid going past 280)
Note directions are as: Looking down on sid clockwise is +ve azimuth

Fix: 
If sid reaches pre-lim limit:
  Stop tracking immediately.
  Drive sid back from limit with 'Abs Az' or 'Rel Az' in sidcongtk
If sid reaches pre-lim-lim limit:
 Note a few things:
  1. The error message will tell you whether its a cw or ccw limit
  2. The anti-backlash string (out at the siderostat) will indicate whether its 
     a cw or ccw limit too.
 In sidcon server (not sidcongtk):
  -Set mode to acquire (even if already apparently in Acquire mode)
  -Set az velocity. +ve to go cw or decrease az value. Set to 1000
  -Go out to sid and hold limit-switch down to allow movement. The limit switch 
   is the toggle switch on the side of the wall-mounted plastic electronics box.
   (Note: limit-switch in n1 may not be wired up?)
  -Trust me it is moving! Look closely, look very closely. Eventually (a few minutes) 
   it will move out of the pre-lim-lim limit into the pre-lim limit.
  -Release the limit-switch. Return to control room.
  -Index the axes and move out of the pre-lim as above.

Problem: N4, S1 loose tracking and slew wildly when tracking across 0deg azimuth
Fix: Stop tracking in sidcon, wait for star to pass west of meridian (by ~0.5deg), 
     re-acquire star and continue.

Problem: 0 counts appear over the entire CCD region unless very
bright beams were placed on the CCD.
Taking darks ands/or restarting wobble2 has no effect.
Fix: The bias level for the CCD has mysteriously changed by several thousand counts.
-Edit normal.cfg in /usr/local/susi/bin/wobble2  using CVS
-Under the `CCD' section, change `blacklevel' to a higher number, say 5000.
-Restart the wobble2 software. You will also need to restart wobble2gtk and stMsg.

Problem: wobble2gtk does not start from gaheris and gives error 
 'Timed out waiting for response from server' 
Fix: Start wobble2gtk from arthur. Can start wobble2gtk from any machine.

Problem:
Wobble has trouble taking over from acquisition system.
Fix:
1. Improve origin coords on acquisition system. ie, Open servo, ensure star stays
   in its box on wobblegtk, then set origin on stMsg
2. Reduce image brightness to minimise the spot size.

Problem:
Red box does not drag star onto origin point.
Fix:Check sids are "track"ing. Try 'reopen' in scheduler. or...
Check wobble2. If it says "There has been a broken pipe I can not identify!" then
close stMsg, wobble2gtk and wobble2 and restart them.

Error:
Pathcomp and Fringecon fail to display the same offsets,
ie. "feedback" in fringecon and "Fringecon offset" in opcon are different.
causes a loss of fringes, .
Fix:
Do pd then pc in fringecon.

Error:
After doing an sb 'Actual OPD (m):' reads -inf
Fix:
Do ld ('load defaults')

Error:
I cant see any fringes!
Fix:
What to do if you don't have fringes (or before looking)[MJI]:
1) Check that the beams are aligned. A rough check (even on a faint star)
 is to put refin, go to widefield mode, close one shutter at a time and
 check that the reference beam is in its box.
2) Check that both beams are entering the APDs. I suggest closing one of
 the North and South main shutters, then opening it again and watching the
 fringecon counts go down then up on both apds. This double-checks you've
 got the shutters in the right spot etc.
3) Check that you're searching in the right spot. This means that the LDC
 offset (currently [May2004] input as a user offset) must be correct, and the
 astromod is correct (ie. the correct periscopes/siderostats are set).
4) Check that the pathcomp errors are not too large (can be >~5-10, want <1.0).
  Fix: Just stop(2) and start(1) tracking in pathcomp.
5) Check that the parameters (look-up-table, filter) are correct in
 fringecon.
6) Check that the metrology system isn't playing up. This may mean slewing
 to the zeroset, but checking for red lights is also good.

If all these checks are OK, then you should have fringes, and should
search  for a couple of mm to either side just in case.

Other non-standard problems could be:
A: A mistake was made entering a baseline solution/WLF in the
 baselns/pathcomp_defaults file.
B: The computer's clock is wrong (check that sidclock agrees on a couple
 of different machines, and agrees with the GPS: you can check the ntp
 status with /usr/sbin/ntpq -p).
C: The star you're looking at has an incorrect position in the starlist,
 or is too big (check another star in this case...).
D: The scanning mirror isn't scanning. This has never happened, but is
 possible if someone bumps a cable or something. If you're really getting
 to this stage, then you can check the mon (for monitoring) output of the
 piezo amplifier (on the roof of the enclosure near the APD's with an
 oscilloscope and check that the voltage is ramping up and down.

Error:
'A dispersion block is moving or stuck'
Blocks are right up against the end of the rail.
Fix:
Simply slide blocks to other end of rail then to centre position.
May need to restart LDC 

Error:
Many RTP failures
Fix:
Ensure a fan is blowing across the LDC electronics

Problem:
Fix:

Problem:
Fix:

Problem: Get Lock file errors on starting UI's.
Fix: Check you are starting the UI on correct server.
     Reboot the server: Press reset then login as Observer.
     Attach monitor to Peleas if necessary or use 'Master View' box
     for Gaheris/Gareth.

Problem:
Fix:

Problem: One apd shows zero counts and nothing is blocking it and nothing seems
         to be wrong.
Fix: Wait a while (few hours). Really, for no apparent reason the apd returns
     from its holiday and starts working again. Its happened to me twice [AJ]!

     You could use only one apd. Just set process type to normal. And reduce
     appropriately.

Problem:
Fix:

Error:
OPLC fails to keep position
OPLC fails to return to correct zero position

Check metrology system.
See MI's notes[link] to adjust.
Metrology lasers require a stable ambient temperature (to within ~2deg).
The air-con keeps temp stable enough but while on causes vibrations
 which are visible in your data.
Otherwise laser cavity heater cannot cope => OPLC position errors.

Fix:
Adjust metrology system heating
Keep air-con on at ALL times when not observing - switch off ONLY 
 while taking data in winter, if at all. Keep it on in summer

Error: Metrology signal dropping away on osciloscopes
Fix: Ensure carriage is at zero position first. If not met will loose the
 position and you will have to slew back to zero anyway.
Realign metrology mirrors. Drivers 11,12 for S. Drivers 13,14 for N.
Use pico speeds of  5 to 20 or so.
On osciloscope next to control desk upper trace is S met.

Problem:
Fix:

Problem: There are large offsets (~>500) in fringecon when trying to find
         fringes.
Fix: This is a problem with the siderostat pointing and requires improved
     baselns file data. 
Open the scheduler_log file. Note the offsets for each run.
Enter these offsets into the carservo_offsets array. Run the check_v2 script.
Note the line 'Change to baseline solution from this data (microns): [numbers]'
Add these values to the values in the appropriate line in baselns, be aware
baselns values are in metres.
Restart astromod to pick up the new values.