Malibu Hydro...  System Review Notes.

To main site index




Overview             Updated April 10, 2026.     Additions or corrections welcomed.

Since March 2026, a complete upgrade has been done to the control system of Malibu Hydro. In order to keep this page consistent since it was first published, the new information will be presented in chronological order, so starting lower down this page. There are new pictures showing the new upgrades.

This page is offered to help in the repair diagnostics and upgrade process of the Malibu Hydro System. The first section below provides an over view of the existing system and the recent operational problems that have happened since April 6 of 2025. Included is an explanation of how the entire system works. It can be a bit daunting looking at it as a whole, but by breaking each section into functional blocks, it is easier to understand how it all ties together. The accompanying pictures detail the areas of particular interest. Thanks to all who have contributed by emails or on site analysis.

The plant was first put into service in June of 2005. Despite a few significant shutdowns, it has run for just under 20 years. Since April of 2025 there have been numerous shut downs for as yet, unidentified reasons. Here is the order of events by date.

Over the past winter it was discovered that if the hydro plant was taken off line intentionally, or from ice in the intake, and if it was very cold in the power house, that the SR 498 generator protection equipment would not start up, no activity on the screen or the many led's. It was by discovered that a solution was to warm the unit to room temperature, then it would most often boot up when the plant was restarted. This was the procedure for the last year or more. However, this was not the reason the plant would not start after the April 6, 2025 incident.

April 6, 2025
On a calm, warm, sunny Sunday morning, the plant shut down for no apparent reason. Despite numerous attempts, it would not restart. It is unlikely this was related to the cold temperature issue with the SR-489, but we had no other idea what it could be. The 300 kW diesel was put into service at this time.

April 17-20,2025
The SR-489 generator protection relay was replaced with a new SR-889 unit. However, the hydro would still not start. At that time, it was found that the PK-2100 governor board was damaged, it showed a blown 5 volt regulator and burnt negative traces on the circuit board. Despite the unit chassis being grounded to the main PLC cabinet itself, it still shorted out and did not blow a 5 amp fuse. We have no explanation for this.
Most of the wiring connections and relays were tested, and the hydraulic components were tested as much as possible.

April 25, 2026
An engineer from Canyon Hydro came to assist and spend a few days on site. He did a lot of tracing, testing, and discussions with outside help, but it refused to start. There was an issue with the deflector fail safe valves not closing so the pressure would not build up to move the deflector. We have no explanation for this.

April 27, 2025
A small team of engineers was at the power house to inspect as many connections, components and hydraulics as possible. Tests were done on the signal conditioner card that drives the deflector, all appeared to be working.

May 9, 2025
A new PK-2100 had been obtained and with some IC chip exchanging, it was able to be programmed with the same parameters as the original unit. Note, this was extremely fortunate as the Clifton PID parameters are no longer available. It was wired into place on May 9th, and hopes were high it would start, but still no go. (PID = Proportional Integral Derivative) This is a standard control algorithm used in many machine control applications.
Note, there are two Clifton Labs Z-180 ZCD's that proceed the PK-2100. These (Zero Crossing Detectors) are what provides the frequency references to the PK-2100 for precise frequency control of the system. On the drawings, these three components are referred to as the Z-180 governor.

The PK-2100 is long out of production, so Malibu should invest in another spare or two. It is impossible to operate the plant with out this critical component. As part of the continuing upgrade, a modern and readily available substitute should be considered. It would require programming and integration into the system, and that is not something any of us can do.

May 15, 2025
Later, on May 15th, Canyon was back on site and found a way to get it going by manually sealing off the deflector proportional valve while it was in start up. Once running, the valves sealed without assistance. Despite it now running, the nozzle control solenoid control relays were firing much too often. Relays R-188 and R-189 were activating too frequently, and when they did, they would buzz for a few seconds before actually closing long enough to activate the solenoid and cause the nozzle to move. While it was buzzing, the relay contact were burning from the back EMF arc, despite reverse diodes across the solenoids. The relays were lasting about 3 weeks before being burned out. We obtained twenty spares and change them every two weeks now. This is the most important issue to resolve at this time. Why are the relays firing so frequently and having no effect on the solenoid? Second issue is the surging of the field current, but this does not seem to affect the output.

(EMF, or Back Electro Motive Force is caused when the magnetic field surrounding a coil of wire suddenly collapses. This causes a very high voltage to be developed (induced) across the points of a relay or switch as it opens. We use reverse diodes and capacitors and resistors to reduce this potential to near zero) This is also how the ignition of a car engine works. When the points open, a high voltage is developed in the coil which then appears across the spark plugs.

Various times, 2025
It was noted that at times, the nozzle would not close when the plant shut down. The turbine running indicator would sometimes not clear. The manual says this MUST clear before a restart is attempted. Also noted was the frequency reading on the HMI display would sometimes be wrong. If this reading was being used by the PLC, it would see it as a fault and should shut the plant down. For the nozzle to remain open could be a result of nozzle control Relays R-188 and R-189 having burnt contacts and not making a solid connection. The burning issue is being addressed.

Sept 3, 2025
The original electrical engineer that was involved in the design of the plant was on site to log into the PLC (Programmable Logic Controller) and check the program, to down load the installed version, and hopefully make some adjustments to correct what ever was wrong. The issue of the deflector proportional valve not sealing off was resolved, but we are not sure what code was changed. No further action was taken because the version of code in the PLC would not talk to the old program in the laptop. So the laptop needed to be upgraded.

Oct 13, 2025
We had a new PLC CPU and wanted to program it and install it to confirm it would at least work the same way. But once again there was a snag. The new PLC CPU did not seem to power up when power was applied on the bench. Is it possible it has to be installed in the back plane in order for it to just turn on? If anyone knows, please respond. So leaving that aside, the lap top was connected to the functioning PLC and although the newer code version was able to 'do' more, it would still not permit some values to be modified. So that ended another attempt.

The PLC was returned for evaluation, and later an email confirmed it checked out in a factory inspection. So we have no idea why the cpu card does not seem to draw any current when powered up. In a following very careful bench test measurements were taken as it powered up and it does infact draw a few miliamps, but no front panel indication is is alive. Next step is to install the blank CPU in the system and power it up that way. No telling what will happen as it will have no code installed in it. The plant will not be able to start.

There is one condition which will keep these units from showing any life, and this known as "Dim Awareness". Here is a link to the condition. It explains in detail how to resolve.

https://www.youtube.com/watch?v=HkAkUx0Q9yk

" When you have an new Modicon or a very old one that had a battery die while the power was off, you will end up with "Controller in Dim Awareness ... "

December 12, 2025.
A conversation with a Schneider PLC specialist has confirmed the CPU we use will not power up unless it is plugged into the back plane of the PLC cabinet. So it is not a 'dim awareness' problem at this point. Since there is no code in the CPU and no battery and now power, there is nothing to loose. Next time our Engineer specialist is on site he will try to load the code and run the system using the new PLC CPU.

December 15, 2025.
The plant went off line today and will not restart. The nozzle did not close when it shut down, the deflector did. Controls are not acting normally for some reason now. We will be on site shortly to figure out what is amiss this time. Some new hydraulic components will be installed in the nozzle assembly.

Dec 22 - 29, 2025
Over Christmas we had a few days to work on the plant.

The issue of the nozzle not fully closing much of the time may have beed due to debris in the nozzle. The nozzle was opened fully and permitted to flush out. There was a lot of banging and thumping as debris passed through. Some may have been ice. The main 12 inch inlet valve would not fully close either, so that was exercised and then it closed fully.

Part of our problem may be stuff getting caught in the nozzle and distorting the flow. That is not uncommon. Out intake filter is made up of a steel grid with spacing of about half an inch by two inches. But during the recent flodding, water will flow over the top if the screen and can enter the back side of the screens and work its way down into the penstock. There was a lot of wood around the normally dry side of the dam when I was there on Christmas day, so it shows it was overflowing the entire dam and moving a lot of material.

The solonoid that controls the nozzle was replaced with a new unit to see if that would speed up the response when opening under PLC control. It did work somewhat better. The whole control system reacts differently each time we test it, so it is hard to tell for sure if any one thing is a fix. The nozzle can also be moved by manually pressing on the solenoid unit to force the nozzle open and closed.

There are flow limiters in the hydraulic circuit and these control the rate of movement to limit water hammer or vacuum. These seemed to work alright, and can be adjusted to completely slow the movement or speed it up. We left them as they were. It is suggested they be replaced in any case as they are now 20 years old as far as we know.

The final test done was to open nozzle to 50% and then put the system on line. As soon as it went on line, it lugged down, then sped up, then lugged down and then tripped off. This was tried several times. Camp load was on the 1 meg diesel at the time, and the auto transfer relay at camp switched as it should. But perhaps there was too much load in camp, which averages 300 kW now, that the hydro could not adjust fast enough to pick up the load. Although it always has in the past. We don't know for sure if it was camp load that bogged it down. The plan was to turn off half the load, but we did not get around to doing that.

We found that the wiring order on the nozzle solenoids was incorrect on the prints. BTA had identified that on the power house copy but it was not on the rest of the prints used when installing the diode snubber, on what we thought was the nozzle close relay. No wonder the other one was burning out so much faster. New diodes and an RC network were added so there should be no arcing as the relays open now. Wondering if back emf was getting into the 24 volt supply and damaging other parts?

Jan 2nd we go back to camp with Mike from Canyon to try and fix it so it will run all winter. Plan is to install the new SR-889 relay and program the new plc cpu and see if that will run the plant. As well as test as much as possible to find the existing problem.

Jan 2 - 5, 2026
This time the nozzle and deflector and turbine runner were inspected. The deflector shows a bit of ware on its leading edge but not likely too significant yet. The runner shows a bit of cavitation damage on the back side of the spliter tip. The nozzle looks fine but hard to tell unless it is removed and the curve compared to the design shape. Of note is one of the thick polyethylene sheets that lines the sump has become detached and is folded in a corner of the pit. It has to be repaired because the concrete is wearing away showing rebar and deeper layers of concrete.

The hydraulic accumulator was out of Nitrogen gas so this was recharged from a large very heavy tank we carried to the power house. Mike connected it and charged the accumulator to 800 psi. When the hydraulic pumps run, the pressure rises to 1500 pounds. The recharge seems to have made an improvement to the operation of the nozzle. With the compressed gas, there is constant pressure acting on the solenoids. Left to the pumps alone with no compressed gas it is likely the response would not be as fast. We did notice the plant started up much faster than it has in the past, at least the last few times we started it. No other work on the hydraulics was done.

We then plugged the new PLC into the back plane and this time it came to life. The laptop was connected to the new PLC to load the program code we have on file. However, there was a problem with version differences between some of the code and in the end we could not run the new PLC. When the original PLC was running the laptop could not connect to that either due to one file being different. We have a fellow at Wismer Rallins looking into this for us.

Next plan was to swap the running SR-489 generator management relay with the new SR-889 relay. The new one was programmed with out code from the supplier, but they did not know about the data register map so that was left undone. Mike was working on that but was not able to complete it and test it before we had to leave. For now, the original unit is working, although we question some of the parameters. In particular, we slowed the turbine down and the protection relay did not stop the hydro at the suggested low frequency setting. We want it to shut off at 55 hz if over 5 seconds, and at 50 hz if over one second. Voltage and over frequency limits have to be tested as well.

March to April, 2026

This section details all the work since the control system was extensively upgraded in the spring of 2026. The original equipment was supplied by Canyon Hydro which built the turbine and hydraulics, and Wismer and Rallings Electric who supplied the Schneider PLC control equpment.

Work was done to the hydraulic cylinder that controls the deflector. O rings and gaskets were replaced. A new proportional valve was installed on the deflector control mecanism. Aditionally, the entire HPU unit was replaced with a more modern version. A larger DC pump was included to bring the hydraulic pressure up to the necessary 1500 pounds faster than the old unit.

The new PLC was supplied and programmed by Alaska Automation and uses Allen Bradley equpment. See the new photos in the section below. This new PLC incorporates the governing function which was previously done by stand alone equipment ... mainly the PK 2100 card which is what failed last year at the beginning of all the troubles.

Governing is also performed at the camp site by means of a second PLC which diverts power to a 90 kW water boiler. This still uses the Thompson equipment but a different PLC was substituted. When this governor is in operation, the hydro side governor is not in control. There is no fighting between the two.

A new communication link was installed using more up to date Ubiquity radios which are tied into the main camp network. It is intended that Alaska Automation will be able to log into the system remotely to perform any necessary diagnostics.

More to follow ..... but this is the main jist of the upgrade up till April 10 2026.






Questions that are being asked by the group:
  • How can we check if the cable is leaking to salt water .... we dont have a meger, just ohm meter.
  • We could reduce camp load and see if it can pick up with out stalling. When we did try, it bogged down, sped up, bogged down, then shut off. Never done that before. It was doing this as it slowed down that the fail safe relays were clicking a lot. We do not recall seeing that happen before. They go really wild untill we power the system down. The led's on the PLC output card flash also, so it is something within the PLC or its inputs that are causing the problem.
  • Need to watch deflector and nozzle as load added. Mark was doing that and said some readings on HMI display % open of nozzle and deflector were normal and some seemed not normal.
  • Another issue is if the nozzle taper, or the deflector edge and wear pattern are getting worse, that would completely mess up the position sensing linear potentiometer that provides position feed back to the PLC. The position would no longer correspond to how much water is actually hitting the runner. That would really really mess up the calculation logic the plc uses. I guess I need to go in the pit and check the deflector edge. Scary , have to close main valve!
  • Yes, that April 6th 2025 shut down was really the start of all these problems. When the PK 2100 had the blown 5 volt regulator, and the burnt negative traces on the PCB. I still have no idea how that could have happened that deep in the workings and not blown everything else along the way. You can see on the drawing of the SR489 how the 4160 volt line goes through a PT, down to 120 volt, then directly into the 489 and also to the Clifton Zero cross cards. These are quite well protected with MOV's , op-amps, and a 168 k ohm resistor before going into the PK-2100 , so that is quite good isolation.
  • Hopefully this page helps keep what we are doing and the train of thought accurate.







  • The topics below are listed in order from the alpine lake to final distribution onto the camp grid. If any one of these stages has a problem, then the power to camp could be compromised or be unavailable.

    Lake:
    Fed by rain and snow melt, it has never run out of water in 20 years of operation, but we do sometimes limit use in dry summers. See 'Quick Facts' page for area and water storage.


    Lake Weir:
    A two meter high concrete structure used to prevent the lake draining too quickly and to provide sufficient storage. The main issue is logs jamming the sluice valve, or the valve stem gets bent and damaged. A floating log boom could be built at the entrance to the bay that narrows to the weir. Removable boards could be added to the notch in the weir to slow the discharge in early summer. In very dry summers, the lake does get low, but has never run out of water. It is necessary to maintain the electronic monitoring equipment at the lake. This radios water level information four times a day and posts it on the web site.


    Intake:
    Has performed very well other than submerged filter issue in fall, frazil ice in cold winters, access is difficult at times. Could the intake screen be modified to use a 'coanda' or aquasheer screen?


    Local Power:
    A 100 watt hydro generator supplies power for the intake SCADA and cameras. It runs for years with out attention. The main issue is the water filter plugs up from small debris, especially in late summer / fall. The filter could be changed a small 'Coanda' type filter off to side. A coanda filter is a small panel of horrizontal triangular rods that is self cleaning. Also called an Aqua Sheer filter. They can operate for a very long time with out any maintenance. The turbine is good for years at at time. A small shelter on site would be handy. This could house a few tools and also the electronics involves with the SCADA system that controls cameras and reports water levels and temperature and battery voltage. (SCADA stands for 'System Control And Data Acquisition.')
    There is not enough sunlight in winter for solar power to be effective.


    Penstock:
    The upper section is made of 40 foot lengths of 1/4 inch wall steel roll grooved pipe with victaulic couplings. Most of this is buried along side the access road. Some sections are exposed where water erosion has undermined the pipe. This could pose a risk if a long section with a coupling was unsupported, the pipe could become uncoupled. The ensuing flood would be significant. There is no automatic shut off at the intake pond.
    Part way down, the pipe transitions to 3/8 inch welded steel. It hangs off the hill side, with one significant expansion anchor joint part way down the natural 'shoot' the pipe line follows. Near the bottom the pipe is fully buried under a steep bank, then along the road to the power house. There is one spare section outside on the road that can be checked for exterior rusting.


    Anchors:
    Most of the pipe is buried which serves as an anchor. Not sure if there are other thrust blocks along the route, but there are no sharp bends so it is unlikely. The only anchors are where it leaves the dam and way down the hill there is an expansion joint bolted into rock. Then it enters the power house foundation. We do have the survey plan for the penstock and it shows all the details.


    Penstock pipe corrosion and cathotic protection:
    The penstock is the second most expensive part of this hydro system. The undersea cable the most costly to replace. The steel used in the penstock is a standard grade for this kind of application. It is uncoated in or out so is subject to rusting from both the water and the surrounding soil. It is vulnerable to attack by iron eating bacteria, causing rust nodules to grow on the inside of the pipe. This can seriously impede water flow resulting in greater head loss. To date, there has not been a penstock inspection.

    The outside of the pipe will rust from the same bacteria if present, and from normal oxidization like any iron buried in the ground.

    There have been discussions of adding cathodic protection to the where by an impressed current is connected to the pipe and sacrificial anodes placed at intervals along the pipe and connected by a wire to the power supply, either at the power houses or additional ones spread along the pipe line.

    Small samples of tin can hang in the shop office and show the effect of a 1.5 volt AA battery connected to two tin cans buried 50 feet apart in the ground and left alone for a winter. The effect can be astonishing. To do it to a 4700 foot long penstock would be a considerable effort. Some questions are, would it be worth it, cost effective, prolong the life sufficiently, and be reliable. It is suggested that if the upper section of the pipe line needs replacing, that it be done with increasing schedules of 12 inch HDPE plastic. This would likely be less expensive to purchase, and certainly much easier to install. And it will last virtually forever.


    Access road to intake and lake:
    This is always in need of repair. The old logging roads are prone to washouts due to clogged culverts and poor construction long ago. Fortunately, Malibu has been able to keep up on periodic maintenance and the road is drivable for a quad or side by side vehicle. Recent logging activity has greatly improved about a third of the route, but the road now has huge dug outs for protection against wash outs. These are just barely passable in the quad.

    The route the pipe traverses is protected with land tenure or an easement, so it is safe from industrial activity. But the access road network is not, and sometimes during active logging arrangements have to be made to travel on part of the road. To date that has never been a problem, the loggers are very obliging.

    It may be prudent to further secure all lands Malibu Hydro equipment occupies. The under sea cable is fully registered.


    Penstock drain method:
    A one inch, high pressure pipe and valve outside the power house can be used to drain the penstock into open air, or to route pressure to the turbine side of the massive gate valve. This is necessary because it is nearly impossible to operate the gate valve if the pressure is not equalized across the valve.


    Penstock transient response:
    See analysis by Larry Clifton. This is important with regard to the operating rate of the nozzle. Excessive travel can cause significant transient pressure surges in the long penstock.


    Water hammer:
    Similar to the above. The pressure remains at 550 psi most of the time. It will go as high as 700 psi if the nozzle is closed too fast. There is an adjustable constrictor in the line to limit flow.


    Nozzle:
    Concern is regarding wear and change to shape of nozzle bulb and therefor the flow based on position. This will throw off the calibration made by the position transducer and confuse the deflector positioning. There is a complex mathematical formula relating the shape of the nozzle to the flow through the nozzle orifice.


    Runner:
    The only issue with the runner is damage from cavitation from the deflector distorting the high pressure water stream. Occasional repair to the runner is necessary and there is a spare runner on site. Presently at the landing. The Thomson ELC was installed to reduce deflector insertion and thus reduce cavitation on the runner. The ELC also increases plant efficiency by doing governing control at the camp side.


    Deflector, wear and replacement:
    Over time, the act of deflecting the high pressure high velocity water stream causes wear on the deflector knife edge. This then disturbs the water flow and causes cavitation on the tips of the turbine leading edges. This is repairable at a very high cost. For this reason, and to avoid wasting power, the Thomson Electronic Load Governor was installed. This is not involved in the present issue with the hydro system as it remains off line for now.


    Bearings:
    Many years ago the main bearings failed on the generator. This was a result of a manufacturing defect to the shaft and was corrected by the generator maker. It did however take months of operation on diesel. Later, the end bearing at the exciter end failed and caused another prolonged shut down.


    Hydraulic components:
    Larger micro hydro plants need hydraulic control as speed governing with resistive load would be impractical. Malibu uses water control to the turbine limit the total power produced. The nozzle is the first method of controlling flow, then the deflector splits off a varying percentage of this flow to maintain precise frequency. This method is somewhat wasteful of water in that about 15 % is deflected. Another issue with deflector control is it causes cavitation to the nozzles pure stream which causes erosion to the deflector and turbine tips.
    To limit this damage and to conserve water, Malibu installed an electronic load control governor in recent years which does the fine governing at the camp side by using resistive elements in hot water tanks. This is a significant saving in water usage as camp heats all its water by this means. The deflector is only in stream a few percent now. It can not be totally out of stream due to requirements of the control process, known as PID control. A byproduct of this is known as 'integrator windup' and would result if it was completely out of the flow. In essence, the controller would saturate and have no more room left to effect control.


    Inside tail race pit:
    This was found to be eroding the concrete from the blast from the deflector, so the pit was lined with thick sheets of hdpe plastic. From the tail race pit the water exits on a 30 inch diameter pipe to the river.


    Generator exciter issue:
    One of the major shutdowns was a result of the exciter end bearing failing. The rotor was making contact with the stator. Particular fault codes were generated as a result. Repairs were done off site.


    Temposonic Nozzle Position Transducer:
    Attached directly to the nozzle, it outputs a voltage range of 0 to +10 volts proportional to its position. The signal appears to go directly to an analog input card in the PLC. See the ladder logic to understand how it works. It acts like a long linear 'volume control'.


    Temposonic Deflector Position Transducer:
    Attached directly to the deflector, it outputs a voltage, range 0 to +10 volts proportional to the Parker BD-101 signal conditioner card, see below.


    Clifton Z-180 Governor signal conditioners:
    There are two identical Zero Crossing Detectors that feed very precise frequency signals into the PK-2100 processor. Together these components are referred to as the Z-180 governor. This controls the frequency of the plant.

    This is an important comment in the PLC ladder code, pg 18:
    Quote. " In this plants function, the operation of the deflector is controlled by the Clifton Governor equipment. The only PLC control to the deflector is to energize the two deflector quick close solenoids to prevent the hydraulic fluid from bypassing the deflector proportional valve. To engage the deflector, the two solenoids are de-energized."




    PK 2100 control card:
    This is the component that failed in April 2025 for an unknown reason.

    There are a number of inputs to this card. Two of them are from the Clifton ZCD's mentioned above. This provides the exact frequency the plant is operating at. In addition, there is an internal clock reference that provides the reference frequency for the plant.
    The difference between the clock frequency and the actual frequency produce an error signal which is amplified and (somehow feeds an 8 bit DAC), then this DAC supplies a 0 to 10 volt signal to the next block in the control loop.

    The PK-2100 assembly is also fed digital control signals from the PLC to directly affect the generator. Inputs for RPM, raise RPM, lower RPM, synchronizing, speed governing, and stop. The stop input is the main 'Unit Run Relay R-181' and this is tied into many of the permissives which must all be positive in order for the plant to operate. The raise and lower RPM inputs (may) be how the DAC is first loaded with the the necessary binary number to approximate the correct frequency. (looking into this)

    Since this is an essential component, and the original has already failed this year as noted elsewhere here, it is suggested that another spare be obtained. It is also and long out of production, but stock is still available. Clifton Labs is not in a position to offer repairs to this component. Since the original card had suffered significant damage, it may be worth while adding more protective fuses to its inputs and outputs and power supply. For the main 5 volt regulator to have failed open, and for negative pcb traces to have been burnt that would take an unusually high current. We have no idea how this could have happened. See photo below.


    Parker BD-101 signal conditioner card for deflector positioning:
    This is another critical component and failure would shut the plant down. It operates independently from the PLC.

    The Parker Hydraulics 'BD-101 Signal Conditioning Card' takes one input from the deflector position transducer and another signal from the Z-180 governor assembly, specifically the PK-2100 cards DAC output. (Digital to Analog Converter) These voltage signals are compared and processed and then and output error voltage with a range of -10 to +10 volts is fed to the deflector position proportional valve. This then directly controls the deflector, which precisely controls how much water hits the turbine, there by controlling the frequency of the hydro plant. As camp load changes, the resulting small changes in these control signals adjust the amount of water impacting the turbine runner. Essentially, it forms one large control loop.

    The BD-101-24 is long out of production. Used or refurbished stock is available on line in the price range of $1,500 to $3,500 Cdn. There is likely a modern substitute. That is worth looking into.


    Thomson Turbine, TTG 600/601 governor:
    To reduce cavitation damage on the pelton runner, the Thomson electronic governor was added in (about) 2015. This reduces the deflector insertion in the water stream by linking the generator output power to a 90 kW load bank at Malibu camp. Instantaneous continual communication by microwave radio links both components.

    Over the past two years there have been unusual surges on this system, sudden ramp up and down of the load bank in camp with no explanation of the reason as the base load in camp was very steady most of the time. We are in the process of investigation, but it is quite possible this is the result of an issue at the power plant, ... an issue that is now causing frequent outages. The Thomson equipment has been disconnected since April 2025 as a precaution, but there are still surges at the hydro plant. The generator field meter jumps to twice is normal current for brief periods.


    Paladin level converters:
    For converting voltage signals to 4-20 mA current signals and vice-versa. These are involved with the nozzle position transducer and feed to the PLC. They are located in the main PLC cabinet on lower left side. They have so far, not given any problem, but should be checked non the less. Camp does not have any spares.


    HMI quick panel:
    The main view screen in the power house that shows most relevant data. This has been replaced one about 8 years ago. Turns out it was not the screen, but the Ethernet card driving it. So we have a spare HMI panel that could be installed at the camp side. It would communicate on the existing, or a different, microwave link.

    Since this can only be viewed while at the power house, there is a web camera looking at the screen which is accessible at camp as well as on line via FTP a few times a day. The mechanical 'finger' is the means to change screens remotely, either from camp or from anywhere. However, it is no longer possible to access camp from outside the network due to firewalls etc. The camera FTP's directly to the web site, it does not depend on the shop computer running. But as long as the computer in the shop office is running, another FTP program will send out a listing of the RTU conditions. This way outsiders an keep an eye on operational conditions. At this time (Nov 2025) none of this is working because the SCADA battery in the power house failed and has been removed.


    RS485 card:
    Fits between the SR489 / 889 generator protection relay and the PLC. Communicates via Modbus.


    Ethernet card:
    Fits between the HMI and the PLC. This has been replaced once.


    PLC     (the brains of the system):
    This uses a Schneider Modicon E984-258 CPU, and consists of the main CPU card, four DEP-216 digital input cards, four DAP-208 Digital output cards, one CNOE-211 Ethernet card, and two ADU-206 analogue input cards. These are still available on line as new, used or reconditioned. It is highly recommended to have sufficient spares on hand.
    We have replaced several digital output cards and an Ethernet card.
    A spare new CPU card was ordered this year. To date we have not been able to get it to power up to program it.

    The original Electrical Engineer has been to site several time to connect our laptop to the PLC to extract the working program and inspect it. Effort has been made to make a few changes only to test operational functioning. To date, he is unable to effect changes, likely due to some version differences. It is essential that we be able to program a new CPU and have it function perfectly. There are lots of spare parts, either new or used, available on line and from Radwell, a Canadian supplier of industrial electronics.

    Discussions about eventual replacement of this PLC are on going. There are 20 years of use on the present equipment. New PLC cards should perform as long, and there seems to be lots of used stock available on line. In addition to the active components, there is the main control cabinet frame with a lot of wires and terminal connectors that will not wear out but we do go through and tighten and check all wire connection points.

    Looking ahead, newer equipment should be considered. If the program could be kept as is, or translated, that would be necessary unless it was an entirely new operating system. There are a few other essential bits and pieces for which there should be spares while available. These include the Z-180 Clifton governor, the PK-2100, a full set, or two sets, of spare PLC cards, the generator field controller, the main DC power supply for the entire plant, but this could likely be any generic 24 volt power supply.


    SR 489 Generator protection relay:
    This significant device monitors all the RTD (temperature sensors) from the generator core. Also it monitors frequency as measured by a magnetic pick up in the generator exciter, and total power measured by nozzle position and via CT (current transformers) on the final output stage before it enters the under sea cable.

    The '489' had caused problems in the winter of 2024 when we found it would not turn on when the power house was very cold (the result of ice in the intake). A work around was to heat the unit in the exhaust of a Honda generator, then it would often boot up when the plant was restarted. It turns out that electrolytic capacitors in the units power supply were know to fail. Some of these were replaced and the unit is now functional.

    Malibu did purchase a newer version, SR 889 and this was installed in the Spring of 2025 in hopes it would solve the shut down proplem, but it did not solve the problem. It was later replaced by the '489', but with mixed success.

    The electrical engineer from Canyon was on site twice, and was able to kind of 'jump start' the plant by locking on the fail safe relays on the deflector proportional valve.


    Control relays:
    These relays fit between the output of the PLC digital card and the hydraulic solenoids. They are standard small 'ice cube' relays and last a very long time. However, for some reason, the relays that control the nozzle solenoid are subject to significant arching and are burning out much too frequently. Despite having reverse diodes across the solenoid. Shop test confirmed voltages in excess of negative 180 volts when the relay opens with out a diode. With a diode, the back emf resulted in minus 0.7 volts as expected, but still resulted in a significant arc in the relay. The addition of a 0.47 uF, 200 volt non polarized capacitor was added and this eliminated all arcing.

    It was also noted when watching all the relays, that when ever one relay turns on then off, the LED indicator light on a number of adjacent relays briefly flashes. That should likely not happen. Theses control relays coils are driven by other smaller relays on the PLC digital output cards. Close inspection suggests that these small relays are well protected from high reverse EMF when the coil releases. Could some of the troubles with the hydro plant be a result of poorly protected relays?


    AC switch gear:
    Lightning protection Suppressors in the high voltage cabinet in the power house do show signs of damage from arching. Lightning does strike the ground in severe storms. It is suspected that this may also have been responsible for the damage that happened in the spring of 2025, even though there was no storm activity the Sunday morning it happened. More information is available from camp staff on this matter.


    Cable across Jervis Inlet:
    This is a major component to the system, perhaps the most important and expensive. Camp use to have it inspected every year or two by an under water submersible out of Campbell River. They are looking for corrosion and any sign of wear from rubbing against rock. Especially on the far side where the terrain dips much more steeply than on the camp side. Despite being 'dropped' on the original installation, the cable has performed perfectly. It has been tested for any electrical leakage in the past few years.
    Deepest point along cable route is 900 feet. See 'Quick Facts' for full details.


    Radio link from power house to camp:
    The original plan twenty years ago included a data link to camp to drive a second HMI. This was never implemented as as a cost saving means. As an alternative, a simple scada was added at a later date to provide some basic functions such as head level in the dam and lake, TV camera control, and battery voltages and temperatures at the intake dam and lake.
    Later, when the TTG-600 Electronic Load Control equipment was installed, it called for a fast wireless Wifi link between the two control PLC's. A pair of Ubiuity radios operating at 5.8 Ghz was installed. As a back up, there is a parallel system using Tranzio radios. These are linked together at the powerhouse but only one selected at camp, to prevent a loop condition. These are in continual use when ever the TTG equipment is on line. These links could carry the HMI data simultaneously.


    CAMP SIDE


    4160 to 600 Volt Transformer:
    This sits outside and has not had any problem since day one.


    Switch gear:
    Most of this is looked after by camp staff and specialists. It is not involved in any of the issues with the plant.


    Load Management:
    Since the mid 1980's there has always been a need for load management to prevent the diesel engines from being overloaded. The first iteration was a crude three step SCR controller that managed air heater loads in the power house. By 1992 this was upgraded to control the electric dryer elements in the laundry. This ran for many years controlling the dryers, and with other relays taking signals from the managers 'total camp load' output. When the hydro was operational, there was less need for management as there was so much excess power available. But as Malibu was expanding, the need for load control was once again a pressing issue. In 2016 a much larger, 28 step load manager was installed which now controls all dryers, water tank heaters, and various space heaters. Loads can be set to automatic control, manual control, or turned off completely.

    This manager sums the power on all three phases, rectifies to a precision DC level, then presents that to a microprocessor analog input. Another input monitors the power being controlled by the Thomson load control governor, and two more inputs are for setting the add load and dump load set points. Loads are only added when the Thomson has sufficient load to release. This reduces the hydro plant governor from reacting to camp load.

    The manager is always clocking every few seconds to see if there is a load it can energize. It adds load if the total camp load is below the adjustable set point, and it drops load if the camp total load is higher than the adjustable set point. It has been operational for ten years with no issue. There is a spare microprocessor and other vital parts in stock at camp.


    Thomson Load Control / Governor:
    One of the ongoing issues with deflector control hydro plants operating under high head is the issue of cavitation produced by deflector control. This was wearing out the pelton turbine runner as well as the deflector leading edge. The way the plant was intended to operate was to be constantly dumping 50 kW worth of power by means of the water deflector. This is not only wasteful, but it is hard on the equipment as well as the turbine pit. Much erosion was occurring to the concrete in the sump. This was mitigated by lining it with thick sheets of HDPE plastic.

    To reduce wear on the runner and to make use of the wasted energy, Malibu comissioned Bill Thomson of 'Thomson Turbine Controls' to supply a load control governor that would be installed in camp. This unit would greatly reduce the deflector intrusion into the water stream, as well as control up to 90 kW of load which was all used to heat the camp domestic water. Since its installation, it has done as intended and now works in conjunction with the 2016 load manager unit.

    There have been issues however. This is mainly in the form of hunting and some erratic operation. Most of this is a result of burned out water elements, causing the governor to have to adjust loads controlled by solid state relays or three phase contactors. Rather than smooth increase or decrease in load, this results in larger than intended sudden load changes. This can be corrected by replacing bad elements.


    Water heater dump loads:
    This is a large water tank with 90 kW of heating capacity. In order for the Thomson Electronic Load Controller to function properly, the six, 240 volt three phase elements must be replaced when ever one burns out. Each unit consists of three elements for a total of eighteen, 240 volt, 5 kW elements. They are all needed for smooth operation. The water in this tank, as well as several other electric tanks, flows through heat exchangers in a continous loop. The exchangers then heat the potable water used by camp. What seems to be an issue is, the primary loop is filled with well water which causes what looks like calcium buildup on the elements. BTA suggested that this is insulating the elements and causing premature failure. It has been suggested that this high temperature loop be filled with creek water which has less mineralization. A more recent suggestion is to fill the loop with rain water which can be collected off the metal roofs and piped into the pump room. One good rain event would be sufficeint to fill the entier heating loop.



    Diesel generators:
    As back up, there is a 300 kW unit and a 1 MW unit. The 1 Meg is set to auto start if the hydro drops. Usually the 300 will be put into service and the 1 MW turned off. There is a high cost to operate the diesels. Automatic transfer switch gear engages with in a few seconds of a power outage at the hydro.


    Funding and Scheduling:
    This is beyond the scope of this review.


    End of list





    Detailed explanation of how the control system works.

    Generation:
    This is the purpose of the entire hydro plant. The potential energy in the stored water is turned into kinetic energy as it gains pressure flowing down in the penstock pipe until it enters the nozzle assembly. Here the velocity of the water increases significantly before it impacts the turbine runner. This rotational motion is then converted into electrical energy inside the generator. It is the job of the control equipment to maintain the generated voltage and frequency within precise limits. This particular generator operates at a line voltage of 4160 volts without any transformer at the station side. At the camp side, this high voltage is reduced to 600 volts, and then down to 120/208 volts for local use.


    Power Control:
    This is controlled by the nozzle and how much water is passing through the turbine runner. The nozzle has a very precise shape and as it opens, the amount of water passing through it increases, and therefor the power produced. The nozzle has a linear transducer attached to it, and this produces a voltage proportional to how far open the nozzle is. As such, this voltage should be an exact representation of the amount of power being produced. This is then fed into the PLC for processing.

    Elsewhere in the system, at the final output before the power line, there are current and voltage transformers attached to the exiting power line conductors. These sense the exact voltage and current and these levels are fed into the SR-498. The two inputs should match in terms of power measured. Any difference generates an error signal which is used to position the deflector. (more study of the finer details ongoing)

    It has been suggested that this calibration may be less than ideal, and as a result, the system is constantly trying to correct which results in its hunting and firing the nozzle solenoids to frequently. This needs more investigating as it is the sort of thing that could be causing the problems of late. One possibility may be related to the failure of the capacitors in the SR 489. A number of these had to be replaced in the power supply to keep the unit operating. If other capacitiors in the electronics were failing, this could affect sensitive signaling which in turn is fed to the PLC via the ModBus link. The obviouse solution to test this theory is to replace the SR-489 with the new SR-889 we have on site.


    Frequency control: (Revision to be confirmed by Canyon)
    The actual RPM control of the generator is the most complex part of the system. The frequency of the generated power is sampled at two points by potential transformers. These low voltage AC signals go to the SR-498 as well as to the Clifton Labs Z-180 Governor equipment. Redundant 'Zero Crossing Detectors' sample the AC signal and feed it into the 'Z World' processor card. This card has an accurate high frequency reference clock which generates many pulses during each zero crossing. It is actually counting period, not frequency. So in one interval between zero crossings, there will be many clock cycles generating a sufficiently large number of counts. If there is any difference between a fixed reference number and the actual number of pulses produced between zero crossings, an error signal is produced. This would be due to the generator frequency being too fast or too slow. This error signal is most likely in the form of a voltage which appears to be converted into an 8 bit binary number. That binay number is loaded into an 8 bit DAC chip that then produces a current output proportional to the binary value. That current output is then converted to a voltage and then fed to one of the two inputs on the Parker BD-101 Signal Conditioning card. The other input comes from the deflector position transducer.

    The BD-101 then generates a voltage ranging from -10 to + 10 volts, and this is fed to the Deflector Proportional Valve which controls the exact position of the deflector in the water stream to a very precise position, and there by the RPM of the generator and hence the frequency of the entire system.
    The PLC firmware is written in ladder logic and although hard to understand all the functional blocks, it is very well documented and describes how the control function works.


    Protection:
    This is the main function of the SR-489 relay. It monitors generator temperatures, voltages, currents and will output error codes to the onboard display as well as control signals to the PLC via modbus. There are a lot of things that have to be just right for the plant to operate. This can be a curse at times, but for the most part, it is good it is so well protected. We just need to know how to determine what may be wrong in a shut down. A well detailed flow chart, previous fixes, and a general understanding of its operation is a good start.





    Pictures to accompany text.

    New 2026 upgrade photos below.




    Schneider Hydro PLC.

    Inside PLC cabinet.

    Clifton zcd's and PK-2100 in 2013.




    Burnt traces pn PK-2100.

    PK-2100 governor layout.

    PK-2100 set up.




    Nozzle solenoid.

    Comparator PCB drives proportional valve.

    Proportional valve drives deflector.




    The hydraulic pump system.

    SR-489 protection relay.

    Swapping SR 489.

    Relays that burn.




    Paladin signal converters

    Deflector control hydraulics

    Expansion joint on penstock




    12 inch roll grooved pipe,

    Penstock enters power house.

    Intake dam in first year.





    WiFi Connection to power house.

    Right click and open in new tab.

    The three RTU's.




    First 1985 load manager

    1992 load manager

    2016 load manager




    2016 Load Manager

    Data logger showing varied power use.manager governor

    Moon set over Frankenstein. It's not all about wires and pipes.





    Power house upgrade in progress.

    HPU removed for new.

    New controls at power house.



    Allen Bradley PLC at power house.

    AB PLC governor on camp side.

    One of several HMI screens.





    Field windings from generator removed for inspection.

    Sludge forms on governor elements from Calcium in water.

    Sludge from elments in bottom of tank was removed.







    From Email exchanges.


    An overriding theme from the most recent emails is to incorporate some means of self diagnostics on a kind of dash board that would report any abnormal operational condition. Additional sensors and updated code and wiring would be required to gather this information to a central processor. Information such as valve position, switch positions, permissives, and then the more complicated things like low level signals from the transducers, relay status, and even control registers in the PLC. Some of this would be visible if the PLC was on the network and able to send data to an appropriate terminal. That is potentially risky however. A wrong entry could be rather ... unfortunate, so appropriate security would be in place.

    While it would be nice to have every bit of data available, the SR-489 does already provide a lot of this information on its LCD display and front panel indicator lights. To be honest, the writer has never looked at this in detail for fear of messing the system if a wrong button is pressed. Obviously, a course in how to use the 489 relay is long overdue.

    The HMI also displays a lot more information than what the remote finger will page through. Again, that is not something any of us has mastered.

    If we knew how to use these two instruments better, we may find a lot more hints as to what condition the system is in.

    The more complex aspects, like the low level signals between the Temposonic transducers and the governor equipment .... that would be harder to intercept and interpret. But if any of that failed, the plant would just shut down. Then testing for these signals could be challenging, as we found this spring with Canyon and other on site poking around.

    What is likely the biggest help would be to have a very good understanding of how the entire system works. Up until now we would look at the signal conditioning card and have no real idea that this was what drove the deflector or what the input signals were derived from. Now we have a better understanding, it is much easier to test for what should be appropriate signals. Then tests can be made to determine what does work and what does not work.

    We all like the idea of a big, one line diagram. Something that shows the overall flow of all signals with out too much detail as there is in the schematics. Include expected voltages or signals at key points. Perhaps an overall diagram, then a few pages detailing specific parts such as the governing routine and nozzle control. These actually are the heart of the system and should be well documented. There are a lot of other circuits and wires and switches that, although necessary for the plant to run, are not complicated to understand, but are hard to follow on the many pages. Could some of it be eliminated ..... yes, but since it is already in place and providing protection, it may as well remain. The one thing that did fail within the first few months was the head level sensor at the intake. This was abandoned and a simple fix used two resistors in the PLC cabinet.

    The trouble with a system like this is that when it is off line, there are no signals. Since much of it is one big loop, if any one element is not working, nothing else does. We got around that this year by forcing a few key elements or powering up the system with battery and then manually manipulating transducers and watching outputs. That worked quite well.



    Another strong suggestion is to have redundancy in the system. While it is unlikely Malibu will build a parallel hydro plant or a second penstock or under sea cable, it has been agreed that spare parts should be on site. This is in the works now. And to as much as possible, look after the intake works, penstock and cable. There use to be inspections done on the under sea cable in the early years. There are reports and even video from the underwater rover that did the inspections. There must be records of this somewhere at camp.

    The communication link to the hydro governor and the HMI screen is by ubiquity radio, just off the shelf stuff and easy to replace. There are spare radios in the shop office. Live internet is available in the power house, although the present access point is ancient but phone calls can be made using it. This link can be used for any new monitoring as in our discussions.

    Voice communication to inside the power house is marginal with the hand held radios. There is an ancient base (old Beyond radio) radio connected to an outside antenna. This radio is so old the audio quality is bad. A better radio would certainly be a big help. It is powered from the SCADA battery.

    The SCADA link was a project that was put in place long ago since this aspect of the project was deleted by the then project manager for some reason. The radios are nothing special, 400 meg UHF links used rather than 900 or wifi. This was to ensure we had good communication to the lake RTU as it is not line of sight to camp. An upgrade will likely be using 900 meg Spread Spectrum radios as these punch through similar trees on other sites. This was all done at no cost, and nothing has been upgraded, even the computer terminal is an old Malibu castoff. A few dollars spent could significantly upgrade the scada if it was necessary. Future SCADA could be all IP based, but since there are two remote sites, with one on solar power, the UHF radios may remain the best fit as they boot up instantly from sleep as the lake unit spends most time powered down completely. That is why we can not poll the lake data anytime, it only tramsmits 4 times a day.

    A final comment on the design of the power house controls.
    Malibu got a smoking deal on the entire project thanks to the hard work of the JPD crew. As we understand it, the PLC control cabinet was something that Wismer & Rallins (the company that built the controls) had in stock from a project that was never implemented. It was adapted to fit Malibu's need. That may explain why it seems a bit overkill for its present use. That said, it also houses all the high voltage switch gear, breakers, lightning suppressors, and CT's and PT's necessary for metering. That takes up at least two thirds of the cabinet.

    The under water cable was sourced in Germany and shipped through the Panama Canal. This was also a great stroke of fortune because it was apparently an off cut, or left over piece of cable from some European project. The penstock was locally sourced. The company that oversaw the project can likely fill in more details on these matters.


    www.malibuhydro.com ......... for the entire story

    End of document