Topics:

• A brief introduction.
• vertex manuals.
• history/problems
• explanation of vertex error messages (.pdf)
  
• The vertex software
• Daily monitoring of the status
• hydraulic brake on the dome and carriage house.
• azimuth encoder difference (table of encdif plots)
  
• 17jan13 encoder dif plot after enc2 changed.
• 25nov11 encoder dif plot after enc2 changed.
• 23jan12: az encoder dif after encoder 1 coupling tightened.
• azimuth encoder 1,2 failures (pointing error)
• pictures of azimuth pointer, scribe mark alignment
• luis's pictures of the encoders. on the telescope and in the lab.
  
• az,gr,ch encoder corrections/offsets.
• motors:
• motor,amp, compensation board settings.
• az motor torques
• gregorian motor torques
• motor repair log
• Jon hagen's vertex memos::
• Dome weight

Monitoring info

Measurements:

A brief introduction.  (top)

1. The motors are controlled by the DCS . It is an analog control system.
2. The DCS is commanded by the Siemens cpu928 PLC controller. The PLC controller sits in a Siemens crate that also contains the LCU, PCU interface, and i/o boards. The PLC  is programmed with the STEP 5 programming language from Siemens.  Internally the PLC is a 186 computer. The software  runs on a 10 ms update cycle.
3. The PLC communicates with the LCU (a 486 computer running MS DOS. also called a Siemens cp581) via a shared memory that sits on the LCU. This interface is proprietary to the Siemens crate.
4. The LCU runs MS DOS 6.2. It talks to the PLC via the shared memory.  The LCU's task is to take external requests, process them, and then pass them on to the PLC. External requests can come from:
1. A keyboard / monitor connected to the LCU
2. The OCU (located in the control room). It  interfaces to the LCU via a serial port.
3. The  AO computers   interface to the LCU via a dedicated ethernet link.
The LCU contains a ISA bus that has an irig card, hard/floppy discs, and an ethernet card. Onboard the LCU is a 1 Mb silicon disc. Normal  operation of the LCU uses the silicon disc. The system can also be run from the hard disc (switch able from the prom monitor setup).

Daily monitoring of az,dome,ch status  (top)

• Azimuth daily summary plots (.pdf)
• Gregorian dome daily summary plots (.pdf)
• Carriage house daily summary plots (.pdf)

How the status bits respond to commands.  (top)

• The system started out under vme control with the motors on.
• The command sequence was:
• reset
• pwr on (even though it was already on)
• reset
• Program track 3 (dome and azimuth)
• stop
• These commands are the pntpwrdip sequence.
• Plot 1 General status. These commands did not change any of these bits. In particular, the stop command does not turn off drive power (the main contactors remain closed).
• Plot 2 Dome axis status. The only affect is the stop command. It affects:
• BrkRel: The brake release bit goes from released to brakes on.
• drvEnable: The amplifier drive enable bit goes from enabled to not enabled. This occurs because the plc drives the drive enable request to false once the brakes are on.
• Plot 3 Dome axis mode, equipment status. The only affect is that the system goes from program track mode to stop mode when the stop command is sent. The dome contactors do not open on a stop request.
• Plot 4 Dome amplifier status: The amplifier status does not change because of these commands. The amplifiers remain ready and there are no faults generated.

10apr17 - azimuth encoder 1 fails,jumps by > 1000 deg

• 16:xx az encoder 1 fails. system switches to using azimuth encoder 2 for pointing.
• The az encoder 1 value jumped down by about 900 degrees.. It stayed with this large offset
  
• There was a close by lightning strike (< 2 secs flash to thunder) around this time..
  
• system continued to run using az encoder 2 for pointing
  
• I don't know if the operator issued a reset or not..
• 20:00 (about) site loses power, generators come on.
• 20:01:48  azimth encoder 2 fault arrives.
• This was probably caused by the loss of power.
• 20:02:24 az encoder 1 fault is reset
• operator probably entered pntpwrdip.. this does a reset
• The system switches back to using az encoder 1. Az encoder 1 still has the value around -600 degrees.
• az encoder 2 fault remains on.
• 20:11:04 az encoder 2 fault is reset.
  
• the az encoder 2 value did not jump
• 22:xx   osvaldo, willy working on encoders.
• they reset the encoder 1 value and start moving it.
• encoder 2 start to jump around during their testing.
• 23:00 -> 08:00  telescope left stationary
• 08:00 next day
• move telescope to az=270 with the pointer, and they reset az encoder 1

The plots show what happened (.ps) (.pdf):

• page 1: plots for entire day
• top frame: az encoder vs hour of day
• black: az encoder 1 (normally used for pointing).
  
• red: az encoder 2
  
• The blue  dashed lines show where encoder failures (or recoveries ) occurred.
• az encoder 1 jumped around 16:00. az encoder 2 was stable
• bottom frame: az encoder2 - az encoder 2  (in deg)
• the difference jumped at 16:00 (because encoder 1 jumped).
• the stuff around 22:00 is when willie and osvaldo were moving the telescope.
  

• page 2: the azimuth status for the day
• 16:0x .. enc1 flt occurs and stays on till 20:00
• telescope was now using encoder 2 to do the pointing (probably 10s' of arc sec pointing error).
• 20:00  encoder 2 flt, encoder 1 reset.. probably from the power dip.
• page 3: az status blowup 20:00 to 20:12
• 20.03 servo failure.
• enc1flt was already active, encoder 2 had a fault. this was probably from the power dip
• 20.04 enc1 flt reset
• must have been the operator entering pntpwrdip after the power dip. this reset the enc1 fault
• It did not bring the encoder 1 value back from 1500+ degrees.
• page 4: blowup of encoder positions  around 20:00
• top frame: az encoder values vs hour of day
• black enc1, red enc2.
• green line.. power dip.. enc1 goes 1600-> 0 probably since system has no power
• blue line, enc1 reset. system using encoder 1. value=1600deg.
• purple line: enc2 flt is reset.. (not sure what did this).
• middle frame azimuth velocity from amplifier
• at blue line , velocity goes to -slew rate (-.41deg/sec).
  
• The position in frame 1 is decreasing (though it is reading 1600.)
  
• So the telescope is moving.
• At 20.14 the drive is disabled (see previous status page), motion stoppped. This was probably  the operator stopping the motion because of the strange encoder value.
  
• bottom frame: size of steps made by encoder.
• This shows the 1 second differences between encoder values (we read them once a second).
• The units are now encoder values
• I've overplotted  green dashed lines showing power of 2 changes. a few of the steps are close to a power of 2 (a bad bit?).  I think the values read back from the encoders are grey codes..
  

Summary:

• 16:00 encoder 1 jumps by about 900 degrees. this may have been from a lightning strike
• 16:00 to 20:00.
  
• The system switched to use encoder 2 from 16:00 to 20:00 .
• This will introduce pointing errors up to 1 arc minute (depending on azimuth).
  
• The value from encoder 1 remained way off. it  later jumped up by about 2500 deg.
  
• at 20:00 there was a power dip.
  
• enc2 had a fault ( which is normal for a power dip.
• the operator must have done a reset to clear the power dip errors.
• enc1 fault reset, but the large offset value remained in the encoder.
• enc2 fault reset about 10 minutes later.
• enc2 did not have  a large jump in value.
• 22:00  willie and  osvaldo move the telescope manually. The azimuth encoder value finally jumped back to 49 deg.
• 08:00  willie and osvaldo moved the telescope to 270degrees (with the pointer) and reset the enc1 to 270 degrees:
• both the dome and the carriage house were at stow when this was done.
• az 1 encoder read:
• 1872693[5-8]  counts at 270. [5-8] means this digit jumped around between 5 and 8.
• The photograph shows the position of the az pointer when the reset to 270 degrees was  done (.jpeg)
• It looks like some offset was put in the enc1 . I wonder if the encoder has a command that can cause it to reset to a default value? Could the lightning have triggered this?. enc2 seemed to behave a lot better..(although it did jump while  willie and osvaldo moved it.

processing: x101/170411/azencerr.pro

11oct16:when system returns to enc1 after enc1 fault

• enc1 fault appears. yellow on the display
• system switches to using enc2.
• You can tell if enc1 is working or not by looking at the encoder difference printed out.

• move to az 700 since large encoder difference
• trkinit lbw
• pnt tr 700 10.9999 -cx
•   pos output: 700
• Test with failure during system stopped
  
• stop system
• pnt mode stop 7
•  remove enc 1
•  enc1 failure
• motion failure (since large jump in position moving enc1 to enc2
• error is red.
•  position now 700.0463 (coming from enc2 values)
• encoder difference is garbage (since no enc 1)
• pnt reset 15
• gets rid of motion failure
• enc1 failure now yellow ... tells us it is using enc2.
• re insert enc 1
• encoder dif returns
•  position still 700.0463 .. so still using enc2
•  pnt mode pt 1
• re enable tracking mode az
• telescope moves.. since last prog track position was 700 and enc2 was at 700.0463
• pos output now 700.. this is coming from enc2..
• pnt mode reset 15
• clears enc1 flt
• telescope position jumps then comes back to 700.
• it is now using enc1
• Test with failure occurring during system active.
• both encoder connected
• trkinit lbw
• pnt tr 700 10.9999 -cx
• encoder 1 driving system
• pull out encoder 1
• causes enc1 fault and motion failure. error is red
• pos reads 700.0462
• pnt reset 15
• resets motion failure
• enc1 fault now yellow
• pnt pt 1
• go back to tracking mode
• telescope moves to position 700..so still using enc2
• pnt reset 15
• removes enc1 fault
• enc difference now valid. enc1 working
  
• telescope position jumps and then returns to 700 so now using encoder 1.
  

• pnt tr 15
• Or pntpwrdip (will do the same).
  

10oct16: az 1 encoder fault (top)

    Azimuth encoder 1 had encoder faults on:

• 10oct15  12:22 to 12:27
• 10oct15  12:27:54  to 12:28:24
• 10oct15  14:50   to 11oct15 07:00 (when the operator did a reset because of a power dip.

• za=10: .06deg*sin(10) = 37 arc seconds great circle
• za=20: .06deg*sin(20) = 74 arc seconds  great circle.

    We also record the difference between the 2 encoders (for the azimuth bending). If encoder 1 fails, then the difference is fixed  (since the value of encoder 1 is not changing).

    During the azencoder 1 fault,  the az encoder difference was still being recorded. So the encoder 1 faults looks like it was a glitch. The error message gets latched until the operator resets it (so we can see that the error occurred).

    On 11oct16 we verified that after an encoder 1 fault, the system continues to use encoder 2 until the fault is reset. So we had degraded pointing 14:54 10oct16 till 07:00 11oct16.

The plots show the pointing error from using az encoder 2 (.ps) (.pdf)

processing: x101/161011/en1flterr.pro

01oct16: az 1 encoder failure. (top)

• ch a stow, gr at 11 deg.
• moved ccw into 270 degrees.
  
• positioned with track mode: pnt tr 270 11 -cx
  
• Used tracking offsets till the pointer was lined up with the scribe mark.
• Willie then adjusted the encoder offset will we had 270 degrees on the readout
• Note: when he updated the encoder position, there would be an encoder fault (because it jumped). You needed to do a reset to get the updated value output.
• the final values were:

• az	encAt lcu	encoder offset	prior motion
  270	18726920	5999999	sitting
  270	18726923	"	270->268->270
  270	18726920	""	270->272->270

•  
• there was a little jumping around in the last digit.
• Then moved to za=6.5 (jan16 swap) and still had 18726920 with occasional jumps 910->923 (wind?).

    After replacing the encoder and moving it to the stripe, willie took some pictures of the alignment:

• pict 1: looking down the pointer
• pict_2: looking down the stripe on the az ring girder.  You see the pointer at the stripe.
• pict_3:  another look along the pointer.

    We didn't have a ruler against the line so not sure exactly how far it is off. The previous pictures showed that the scribe mark is about 1 mm wide.

    Position error.

• The radius to the scribe mark is about 60 ft.
  
• a 1 mm error on the pointer/scribe mark is :( 1/25.4)/12/60 *radeg*3600=11.3 Asecs /mm (little circle)
• a 1 mm error  at za=10deg = 2asecs
• a 1 mm error at za=20deg=4 asecs great circle
• We really need to align the pointer/scribe to the position that was used when the model was made.
  

    Down in the lab we opened the encoder. The glass disc with the  position info was broken.
This was the encoder that was installed 10jan16.

22jun15: az 1 encoder  failure (top)

    Az encoder 1 failed atround 18:00 22jun15. The telescope continued to track (since it  switches to use enc2 for the pointing).

    azimuth status 22jun15 (.ps) (.pdf)

• the data covers 17:00 to 24:00. The failure occurred around 17.85.
  
• the dashed vertical lines show when the enc1 failure occurred, and when the telescope stopped moving.
• When encoder 1 fails, the system automatically switches to use encoder 2 for pointing.
  
• page 1, bottom is the encoder difference enc1-enc2,. These are actually narrow jumps. so the bad values were intermittent.
• page 3: the motion failure which stopped the telescope motion occurred about 18 minutes after the encoder 1 failure.

    azimuth status (blowup when motion stopped) (.ps) (.pdf)

• This is a blowup around when the telescope finally stopped moving
• page 3: the enc1flt went away (around 18.198). On the next second we got a motion failure.
• My guess is that the system switched back to using encoder 1.. It probably then tried to use the current and previous positions on enc1 to check for motion failure. Since the old values were probably wrong, this probably generated the motion failure and caused things to stop
  

    gregorian dome status during failure (.ps) (.pdf)

• For reference, this shows the position of the  dome

Aligning new encoder.

• The dome was at 6.5 degrees
• We moved the azimuth to 270.0000 degrees using program track mode. then stopped the azimuth. (it was pretty windy).
• The encoder reads at 270 were:
• enc1: 1872690
• enc2: 1872599
• We placed a metric tape measure in above the alignment mark, and then took a picture of the alignment arrow (.jpeg)
  
• from the picture you can see that the arrow is about 1 mm to the left of the mark on the ring girder.

17jan13:az encoder 2 replaced  (top)

    On 17 jan 13 aeronomy was doing azimuth spins when encoder 2 failed. The encoder was replaced.

The plots show the azimuth encoder difference before and after the replacement (.ps) (.pdf)

• Page 1: 17jan13 azimuth encoder difference before and after the change:
• Top frame: encoder units
• bottom frame: degrees
• black is CW new encoder
• Red CCW new encoder
• green CW old encoder
• blue CCW old encoder
• The dome was at  za=15 and the ch was at za=0
• Page 2: 22jan13 az encoder difference with new encoder
• za ch=15, zadome=8.79

23jan12:az encoder 1 becomes loose. (top)

• The motion was:
• dome=10.976 degrees, ch at stow.
  
• 270 to 0 at .4 degrees/sec
• 0 ->360 at .4 degrees/sec
• 360->270 at .4 deg/sec
• Page 1:The black trace is the clockwise motion. The red trace is the counter clockwise spin.
• Top: azEnc1 - azEnc2 in encoder units.
• Bottom: azenc1-azEnc2 in degrees. The radius is about 60 feet so one degree is about 1 foot at the encoder radius.
  
• Page 2: compare az encoder dif with 19jan12 before coupling tightened.
• If the azenc1 moves and everything else remains constant, then the encoder difference will also change. To check this, an az swing during the world day (19jan12) was used. On 19jan12 the come was at za=15 degrees (ch at stow) so it was not exactly the same as 23jan12.
• Top: Az difference clockwise spin comparison.
• black 23jan11, green: 19jan11
• Bottom: az difference ccw spin comparison.
• red 23jan11, green 19jan11.
• Comparing the 2 dates, the az encoder diff has shifted about .01 to .02 degrees. This is most likely from the different za of the dome.
• Looking back on 25nov11 (see below) the dome za was actually moving during the measurement so it probably can't be used for the comparison.
  
• For reference:
• .01deg change at 60 feet is about .12 inches.. this is probably the accuracy we can position the pointer at 270.
• .01Deg az error on the rack gear is on the sky:
• 20deg: 12 arc seconds
• 10 deg: 6.2 arc seconds.
• Need to redo an az swing at dome=15 to compare with 19jan12.
• When ever we do the az swings to check the az dif we should use a fixed dome za (say 11) and a fixed ch za (say stow).
  

25nov11: az encoder 2 replaced. (top)

• The motion was:
• 270 to 0 at .4 degrees/sec. dome at 5 degrees, ch at stow
  
• 0 ->360 at .4 degrees/sec. dome at 5 till  az=110 then dome moved to 10 deg za (got there when az=240)
  
• The black trace is the clockwise motion. The red trace is the counter clockwise spin.
• Top: azEnc1 - azEnc2 in encoder units.
• Bottom: azenc1-azEnc2 in degrees. The radius is about 60 feet so one degree is about 1 foot at the encoder radius.
  
• The az encoder difference for the swings has a negative offset of about 140 Encoder counts.
• To center the difference, azenc2 should have it's value decreased by 140 counts.
• This is probably not critical since i think the az bending limit is set to a value larger the 400 encoder counts.
  
• The PI control loop uses the azEnc1 encoder to control the motion.
• The plots show that the azEnc2 (on the opposite side)  leads the azEnc1 a little (probably some offset in voltages sent to these motors).

12apr09: mot52 fail at az 326 11/12apr09  (top)

• The failure occurred near the same time on both days.
•  This must be because carl was tracking the same source.
• The dome was moving down from 19 degrees at .002 deg/sec and the azimuth was increasing at .01 deg/sec.
  
• It was then commended to slew to a new position. The failure occurred 5 seconds into the slew.
• To replicate the failure:
• pnt tr 325 18.92 -cx
• wait for the telescope to get there.
• pnt tr 327 17 -cx
• this should move it thru az 326 at slew like the failures.

07mar06 azimuth encoder 2 jumps  (top)

• Top: the azimuth position (encoder 1) versus hour of day.
• Bottom: The azimuth encoder difference (azGr - azCh) in degrees.

    On 08mar06 the encoder 2 jumped again. It was replaced by a new encoder on 09mar06

---

20jun02: dome motor amp glitching when changing direction.  (top)

Fig 1 top. Torques versus hour of day. The zenith angle is plotted in black at the top. The green spikes are the torques from motor 21.

Fig 1 bottom. This is a blowup of the top plots. The sampling is once per second so the glitch is on the order of  1 second.

Fig 2. The top is torque versus velocity (degrees/second), middle is torque versus acceleration (deg/sec^2), and the bottom plot is torque versus za. The green crosses are motor 21. The glitches only occur at positive acceleration.

---

11nov05 azimuth encoder failures. (top)

• 11nov05 (.ps): 16:00 enc2 fault. 16:20 enc1 fault. replaced encoder 1 with a spare encoder. Used the az index to align the azimuth arm.
• 13nov05 (.ps): 11:07 encoder 1 fault. Kept running
• 14nov05 (.ps): early morning . encoder 1 (the one that was installed on friday) failed. Moved enc2 -> enc1. put bad encoder on encoder 2.
• 16nov05: replaced encoder 2 with a new encoder.

---

12aug04: dome vibrations while moving at slew rate at low za.  (top)

• Fig 1. The top two plot are the azimuth position (black) and azimuth velocity (red). For the azimuth .4 degrees/second is the slew rate. The bottom two plots are the dome position (black) and the dome velocity(red). The dome slew rate is .04 degrees/second.
• Fig 2. The top plot are the azimuth motor torques for the 8 motors. The red graph at the top is the azimuth velocity. The solid black line is a scaled version of the azimuth position. The bottom plot is the torques for the 8 dome motors. The solid black line is the dome zenith angle while the red lines show the dome velocity.

24aug01: az servo failures. Motor torques build up while stopped. (top)

    There have been a number of servo failures with the azimuth amplifiers recently. The error message is (amp xx not ready). When the amplifier leds are inspected, an over speed condition is present. The problem occurred at more than 1 az, za position but it was repeatable whenever we used az=334.6 and za=2. The sequence that caused the problem was:
1. moving in increasing azimuth (at say .1 deg/sec) close to 334.6, dome at low za so most of the weight is on the wheels opposite the dome side.
2. user switches receivers. the trkinit routine is called and it does:
• pnt x vme 3. This command transfers control to the vme system (even though it already had it). A side affect is that the motors go to stop mode.
• A second or two after this command is executed, a  pnt mode tr 3 command is sent. This requests the az and gr  to move to program track mode.

 
 People were placed near each azimuth wheel when this sequence was executed. We were thinking that maybe the wheels were spinning in this area. They reported that:
• The wheels would slow down and they could hear the brake come on.
• While the brake was on, they could hear a noise in the motor. It sounded like the motor was fighting the brake.
• After about 5 seconds there was a large bang and then the wheel would spin fast and then stop.

 
The Figures show the motor torques and i/o bits during this sequence. The x-axis is the time in seconds from midnight. The data is sampled once a second so transitions can be occurring that are not being plotted.
1. Figure 1 shows the 8 motor torques versus time (about 11 seconds). The torques are offset for plotting. They take about 4 seconds to rise to a maximum value of 20 amps. This is happening with the brakes on. On the fifth second they plunge to zero.
2. Figure 2 shows some of the i/o bits during this time period. The most interesting bits are:
• IvelAz. This controls the PI error integration. It remains on for the entire sequence. Bit 4 of digital output byte 10 (sum_n controller) also stays on so the output of the PI loop is being sent to the motors.
• BrkG48Rel,BrkG15Rel show that the brakes were on at 60982 seconds.
• horn. This went off at 60981 and back on at 60982. This implies that a new request to move was being activated.
• The green line with * over plots the motor torques for az motor 11 during this period.
3. Figure 3 shows the transition from azimuth moving at .1 degrees/sec to stop without a restart request. From bottom to top the bits plotted are: drvsEnabled, brksReleased, then output bytes: Q10,Q8,Q9. These output bytes control the relays that cause different portions of the velocity request to be included in the value sent to the amplifiers. The average velocity is also over plotted in green (*). [note: it bothered me the first time i saw the drvenable going low before the brkrelease had gone low (the drives should never be disabled without the brakes on). After searching the code it looked like the drvenable bit high implied that the drives had finished the drive up sequence (or they had not yet started the drive down sequence)].

 
The time sequence for the failure (figs 1,2) was:
• 60978 - The transfer control command was probably sent. This started the slowdown/stop sequence.
• 60981 - request brakes M11,M12 to be on. The horn was turned off here (this is on when in motion). The DCS az 0 speed signal has become true.
• 60982 - request brakes M51,52,41,42,81,82 to be on. The read back shows that all brakes are now on. The program track request was probably sent here too since the horn has been re-activated (this occurs pending new motions). The torques begin to rise at this point.
• 60986 - the torques reach a maximum of about 20 amps.
• 60987 - a servo failure occurs (over speed). The horn is turned off.
• 60988 - the DCS az I portion of PI loop is turned off.
• 
  
The likely sequence of events was then:
• The azimuth was asked to stop so it started slowing down.
• motors m11,m12 stopped first. The other motors stopped 1 second later. In between this time a  new request to move arrived. The differential stop times may have been because some of the wheels were slipping.
• The brakes are on and dcs spd0 is true. This disconnects the input velocity request in dcs brd 4a1 via relay K7.
• There must have been a non-zero difference between the grounded velocity request and the actual speed value read back from the motors. This value integrated up for the 5 seconds that it took to startup.  With the brakes on, this caused the noise and vibrations that the people heard.  For some reason, the velocity error integrator was not turned off during the 5 seconds.
• When the startup time had completed, the brakes were released. The 20 amps of torque now had 0 load. It caused the loud noise and the motor wheels to spin rapidly. The rapid spinning made the amplifiers go over speed and shut the system down.


The az,gr,ch information is recorded once a second.  I searched the files from april through august and found more occurrences of this problem: 2 in april, 5 in august, 11 in june, 8 in july, 5 in august. I never saw it occur in the gr or ch. This is why  i think that a wheel slippage may be related to this.

I've changed the trkinit code so that it doesn't try to take control of the axis if it already has it. This will cut down on the number of times that this condition should occur. It should still be fixed since users can request the azimuth to stop and start without going through the vme system (via the lcu or ocu).

This problem will stress the motors/amplifiers (removing the load so quickly). The brakes should also be inspected. Jose maldonado pointed out that wheels spinning on the track will remove the grease and could cause damage to the rail.

01sep01 update. Motor/amp 12 on the azimuth had an output voltage of 300 millivolts when the azimuth was not moving. All of the other motors/amps were under 10 millivolts. This value was averaged with the other 7 motors and caused the velocity error to be non zero and integrate up while the brakes were on. Fixing this will decrease the chances of this happening again. The code should still be changed so that the integrator is disabled whenever the brakes are on (at least for the azimuth).

---

History/Problems (top)

• 02oct16: az encoder one fails. replaced and realigned use index at az=270.
  
• 10jan16: az encoder one fails, replaced and realigned..
  
•     look at elec page for failures here
  
• 12jul13: drag link comes loose (pig motors 22,21), connection plate bent on dome.
  
• 18jun13: gregorian mot22 replaced after problems with 22 continued (more info)
  
• 17jun13: gregorian amp22 replaced after multiple failures
  
• 17jan13: az encoder 2 bearing froze up. replaced encoder. enc dif is a little off but probably ok.
  
• 18jun12: azStpCC, servoFail error
• These 2 error occurred together. No other errors were visible. It was caused by the 120 volt breaker (behind the computer monitor rack) tripping. This breaker gives the 120 the motor  amps and to the cable car az limit switch.  The power is pd2208  See pd.22 in schematics .
  
• 26jan12: az encoder 1 replaced.
  
• 25nov11: encoder 2 failed.. replaced.
  
• 23oct11: power dip, amp 81 not ready. lcu parmeter menu encoder offset has 281 deg. Reset to 0.
  
• 09dec08: klaus comes for drive tuneup.
  
• 11jun08: lcu azimuth encoder offset went 1 degree -> 0.
  
• 21feb08: lcu azimuth encoder offset went from 0 to 1 degree. (returns to 0 11jun08)
  
• 05feb08: reset pots on az motor amplifiers to standard position. The cmdScale, current limit of some were not set correctly.
  
• 10aug07: azbending error 9:06am. corner 8 inside painting containment. Grit got into mount spring mechanism that pushes encoder into the rack gear. encoder had come away from rack gear causing bending problem. Manually reset encoder 2 position to same encoder difference as before the failure. cleaned encoder mount.
  
• 08apr07: azimuth encoder 1 fault.
• 04apr07:azimuth encoder 1 and 2 reset.
• 13feb07:amplifier 21/22 goes offline on dome. Amplifier 21 and later 22 of the dome have been going offline when starting to move downhill at high za. This has occurred on 13,15, and 16 feb07.
• 14nov06: az encoder1 fails. replaced. Had to reposition az using pointer.
• 14oct06: encoder board port 1 (az enc1 ) fails. replaced board 18oct06.
• ??sep06: az encoder 1 failed and was replaced (not sure about date).
• 16aug06: aux encoder replaced with newer version.
• 06apr06: ch brakes changed.
• 05apr06: greg brakes m11,m12 replaced.
• 04apr06: greg brakes m42,m41,m32,m31,m22,m21 replaced.
• 10oct05: telescope only moves to smaller az when put in tracking mode: The azimuth 2 encoder was changed in the morning (because it was reporting encoder failure). In the afternoon when the telescope was put in tracking mode, the telescope would only moved toward azimuth of zero. Preset and rate mode worked ok. This problem has occurred before (but i don't have any notes on it). During the morning maintenance, the plc was powered off then back on. After the problem jorge tried a reset and then an overall reset of the plc. The problem continued. He then tried replacing the plc. The azimuth started working again.
• 26jan05 cam limit switch dome: found that the cam limit switch for gregorian is bad. May have a loose tooth or something inside. This generates the hardware version of the pre and final limits (there is also a software version that comes from the encoder). The lower limit was creeping up from 1.06 -> 2.5 -> 3.6 degrees over the last week or so. Readjusted the upper limit to be 19.5 degrees. Should order a new unit.
• 19jan05 vertex shelter overheats: air conditioners vertex shelter tripped off. Trouble with pointing. Only moves in negative az in tracking mode. Turned out that the plc wasn't working (replaced). Other problem was that the negative direction seemed to come from a 1 degree az offset that installed. This comes from f8 params menu in ocu. Gets stored in the file params.dat.. Looking back at old data , this offset has been there since last jun04. With it in now, it is causing the tracking to fail. May be that the old version of the software was ignoring this??. Reset to 0 via f8 ocu. Every time lcu rebooted need to do this. Looks like the new value is not getting set in the file params.dat (we are running on the silicon disc...). Need to figure this out.

<- page up
 home_~phil