Using state-of-the-art computational techniques, a genetic algorithm (GA) and an accuracy-based learning classifier system (XCS) were shown to produce optimal operational solutions for gate structures in irrigation canals. An XCS successfully developed a set of operational rules for canal gates through the exploration and exploitation of rules using a GA, with the support of an unsteady-state hydraulic simulation model. A computer program which implemented the XCS was used to develop operational rules to operate all canal gate structures simultaneously, while maintaining water depth near target values during variable-demand periods, and with a hydraulically stabilized system when demands no longer changed. This model can be applied to canal networks with constant or variable demands within the limits of current hydraulic simulation capabilities. The program output is a set of feasible and optimal operating rules for multiple gate structures, facilitating the automation of open-channel irrigation conveyance systems. Results from sample applications of this technique are presented in the companion paper.
Background and Previous Work
One way to achieve water management improvements in open-channel irrigation conveyance and distribution systems is through the use of advanced technologies to help make decisions to operate gate structures. With regard to improvements concerning the operation of irrigation systems, there are many available models for large-scale water management of conveyance and distribution networks, and there are others that can be implemented at the farm level. Demand-based operating schemes, in their purest form, dictate that water will be delivered at any time and in any amount to users, given some limits, thereby providing great flexibility to on-farm irrigation systems and potentially enabling improved on-farm water management. Farmers and other water users prefer true on-demand operational schemes, but this can make the system difficult to operate and may be expensive, both in terms of hardware and labor. However, very few, if any, irrigation systems are operated with unrestricted flexibility in terms of delivery timing, flow rate, and duration. Nevertheless, any reasonable steps that can be taken to move away from rigid delivery schemes and toward flexible, on-demand, schemes, will help improve water management in irrigation systems.
The aforementioned difficulty for managers is related to the control of hydraulic transients along conveyance and distribution channels during daily operation. Some attempts have been made to automate irrigation conveyance and distribution systems. Automation requires field or model calibration using restrictions generated by operational conditions for each site-specific system. Those restrictions go against operational flexibility, causing delivery schemes to be technically and financially constrained (S. Chittaladakorn, unpublished Ph.D. thesis, 1996). With this approach in mind, this research combined theoretical and empirical information and developed an optimization procedure with specified initial conditions.
A hydraulic model with a mathematical optimization procedure can be used to generate sets of operating rules for canal gate structures. The rules can be custom developed for a specific canal system. One type of optimization procedure is based on classifier systems which use genetic algorithms. As an optimization procedure based on Darwin’s natural evolution theory, genetic algorithms are a powerful tool to solve different types of problems. Genetic algorithms (GA) are based on a principle similar to “the survival of the fittest,” where characteristics of parent rules are transmitted to children by means of reproduction, crossover, and mutation, and prevalence of the fittest rules (those that perform best in meeting operational objectives). Based on natural selection and genetics, Holland (1975) created the original GA theory, and presented the correspondence between rules used in artificial systems to biological chromosomes, detectors or features to genes, and feature values to alleles, among other characteristics. Genetic algorithms are the core for the discovery of new rules and the exploitation of extant rules, and they have recently been used in hydraulic engineering as an optimization tool for reservoir operation for the irrigation of multiple crops (Nagesh Kumar et al. 2006), least-cost design of water distribution systems (Babayan et al. 2006), water quality prediction in water distribution systems (Zheng 2006), design of drainage systems (Peng and Jia 2004), design of composite channels (Jain et al. 2004), scheduling of water pipe replacement (Dandy and Engelhardt 2001), water distribution systems (Simpson and Wu 2001), and location of control valves in pipe networks (Pezzinga et al. 1999).
Accuracy-based classifier systems (XCS) represent a major recent development in classifier systems research. Since its origin (Wilson 1995), XCS has provided repeatable results that are generally better than those produced by the majority of models developed since Holland’s initial work (Butz et al. 2004). Accuracy-based classifier systems represent the knowledge extracted from a problem as encapsulated in a set of rules. A rule is a prescribed mathematical method for taking actions to achieve operational objectives. A rule set is incrementally evaluated by means of interactions with an external environment through a reinforcement learning process, and it is improved by a search mechanism based on a GA (Bernado-Mansilla and Ho 2005). Models of physical phenomena (e.g., hydraulic simulation models) can help an XCS to determine whether an action is plausible or not, by means of a reinforcement program. The reinforcement program assigns rewards and penalties according to system behavior (real or simulated) after the application of a specific action.
Until now, XCS had not been used for developing general operating rules for canal gate structures. However, GAs have been used for decision support in irrigation project planning (Kuo et al. 2000) where relative crop yield and water demand information was applied to maximize the projected benefits. Genetic algorithms have also been applied for optimal seasonal furrow irrigation (Montesinos et al. 2002), where a model produced an irrigation season scheduling for maximizing farmer’s profit, and along the same lines, GAs have been applied for optimizing off-farm irrigation scheduling (Nixon et al. 2001). More recently, a classifier system supported by a GA has been developed for rule-based operation of canal gates (Chittaladakorn and Merkley 2005).
Initial efforts to apply automatic controls to conveyance systems started in the 1950s by the U.S. Bureau of Reclamation, and first steps were published in a canal systems automation manual (Buyalski et al. 1991). Subsequent publications have dealt with canal control using a centralized scheduling logic (Merkley and Walker 1991), designs for the operation of irrigation canals (Reddy et al. 1992), and the design of a global control algorithm for irrigation canals (Reddy 1996). As many canal control algorithms were developed in during the last three decades, some additional efforts were dedicated to organize the information by means of a basic classification (Malaterre et al. 1998), and by an inventory of control algorithms (Rogers and Goussard 1998). Further studies were performed for clarifying control theory terminology for canal control algorithm formulations (Ruiz-Carmona et al. 1998), and doing comparisons and calibrating canal gate automation algorithms (Pongput and Merkley 1997).
Studies have been directed to consider hydraulics related to controllability as in wave-motion stability in canals with automatic controllers (Hancu and Dan 1992), and about transients in canal networks (Misra et al. 1992). Other studies were directed to control and conveyance structures as flap gate design for automatic upstream canal water-level control (Burt et al. 2001), and the influence of canal geometry and dynamics on controllability (Strelkoff and Clemmens 1998). On the other hand, another considered time of opening of irrigation canal gates (Ponce et al. 1999), and local time stepping for modeling flow (Crossley et al. 2003).
This approach has been applied to improve proportional-integral (PI) logic for canal automation (Burt et al. 1998) and more recently a method for automatic tuning of PI controllers has been developed (Litrico et al. 2007). Some studies have been made for fuzzy logic controller for downstream water-level control in canals (Stringam 1998), and to describe the general characteristics of solutions to the open-channel flow, feed-forward control problem (Bautista et al. 2003), and GAs have been used sparingly for canal structure control.
Introduction
This paper provides a detailed description of the elements that comprise an XCS model, a GA, and the hydraulic model used in the study. A companion paper presents results from the XCS model for different operational scenarios in various simulated canal systems. The hydraulic model was used to predict the response from the application of different gate operation rules as generated by the XCS and GA. The response was evaluated through the XCS so as to develop a set of gate operating rules which could be applied to successfully control the hydraulics of a given canal system, achieving specified operational objectives. The principal operational objective was the maintenance of stable canal water depths.
Both trapezoidal and rectangular canal cross sections were used in this study. Structures included in-line gates and turnouts (water delivery points). In-line structures were located at the upstream and downstream ends of each canal reach, and they included rectangular gates and weirs. Turnouts directed water from the simulated canal system to delivery points outside of the system, and they were always located near the downstream end of each canal reach. Turnout demand hydrographs were generated randomly to simulate various operational conditions in the canal system and provide a more generally applicable rule set. Each hydraulic simulation began with all canal reaches full of water and steady-state flow conditions. The delivery of water through turnout structures caused unsteady hydraulic conditions and the XCS was tasked with developing rules to operate the in-line structures in a way that would best achieve the operational objectives and reestablish steady-state hydraulic conditions. Following a period of changing turnout delivery demands, the rules were supposed to rapidly bring the canal system to a new steady-state condition.
A computer program was written in the C# .NET language for PCs to implement the model. Minimum computer requirements are a processor similar to the Pentium II CPU 398 MHz, and 1 GB of RAM. Data were input and output using a console interface and saved in database and text files.
Classifier System
An XCS was combined with a hydraulic model to simulate the response from a set of canal gate operation rules. The hydraulic model also provided information from which rewards and penalties were computed through the XCS reinforcement program. The XCS acted as a reinforcement learning agent, meaning that it learned to perform a task through trial-and-error interactions with an unknown environment which provided feedback in terms of numerical reward. The definition applied in this research corresponded to a classifier system that learned actions for the adjustment (operation) of gate structures, when it received rewards and penalties according to variations in canal water depth relative to specified target depths, hydraulic stability, and demand-supply deviation, among other aspects of the reinforcement program as indicated below. Those interactions occurred continuously during each hydraulic simulation. The input to the XCS was the current hydraulic state of the canal system, also referred to herein as the “environment.” The output from the XCS was a set of actions which was applied to the canal system, and the hydraulic response was evaluated by means of a reward, a penalty, or a combination of both. The goal of the XCS was to maximize the rewards by developing a set of optimal operational rules for the gate structures. Herein, the definition of an “action” is simply whether to open or close a gate structure, and the magnitude of the setting change (which could be zero if there is to be no gate setting adjustment). After coding for computer processing as a binary sequence, it is defined as the action string.
The condition describes a current situation which is occurring in the hydraulic system. Conditions are sensed from the hydraulic system, to provide input to the computer model. In this way, conditions are created based on information obtained from the hydraulic system by the hydraulic model, and they represent a real situation in a network of canals. Therefore, environment situations are translated to that is known as a condition. Similarly, when the condition was coded for computer processing as a binary sequence, it was defined as the condition string.
Classifiers in this XCS consisted of a condition, an action, and three main parameters. The main parameters were the prediction, the prediction error , and the fitness, each of which is defined in equation form below. The prediction estimates the average payoff that the system expects when a classifier is selected for an operational action. The payoff is an outcome obtained from the system according to rewards or penalties deserved for action application. The prediction is computed as a weighted-average fitness of all classifiers that match the current environment, and which are invoked by the current action. The prediction error gives the average error between the classifier’s prediction and the received payoff. The fitness is an estimate of the classifier’s accuracy relative to other classifiers, and the classifier’s accuracy is calculated from the prediction error . All of these data are stored in a population of rules. The population contains possible solutions to solve the operational problem and each classifier corresponded to a potential solution for one particular hydraulic state of the canal system, which might or might not be evaluated through the process, depending on whether the classifier was selected as an action to be applied to the canal system. As the population grows, the average classifier fitness increases if all the parameters are defined adequately, whereby the population is evolving.
Through rules, conditions and actions interacted with the canal system, which current hydraulic state is known as the environment. In the computer program which implemented the XCS, the condition, action, and environment were coded independently as binary sequences (or “strings”). Rules in the population were made up of pairs of condition and action strings. The three main parameters, as described above, guided the evolution of the process with the help of a GA.
The XCS determines the implementation of actions upon the environment. The role of the XCS is to decide what set of actions the system should take to reach the operating goals, and it selects one set of actions at a time to operate the canal system. One set of actions corresponds to a set of gate structure settings for every gate in the canal system. These settings are applied to the hydraulic system as simultaneous gate operations. There are many potential actions that might be selected for application, but the XCS uses only one set at a time. For one specific condition, there might be many actions according to the number of gates to operate, and the number of different gate settings. All actions are created randomly from pseudorandom numbers.
A schematic view of the XCS general architecture is presented in Fig. 1. The model begins with initializations for the hydraulic state (defined by the hydraulic model as calculated water depths and flow rates at discrete computational nodes along the reaches) of the canal system and generates a random population of rules. The XCS creates what is referred to as a matching rule set (described below), a prediction array, and an action set. One set of actions is selected to be applied to the gates in a canal system through a hydraulic model. Therefore, the XCS relies on a hydraulic model, as well as on the reinforcement program (which has an embedded GA) to evaluate the performance of the entire canal system and to decide whether to continue improving the population or not. When the population matures, there is little or no improvement in its strength and at this time the learning process is complete.
Fig. 1. General architecture for generating canal structure automation rules
Matching Procedure
After initializations, the current population is compared to the current hydraulic state of the canal system (environment) to create a “match set,” which is composed of classifiers whose condition corresponded to the current hydraulic situation in the canal system. The condition parts of the classifiers (rules) are compared one-by-one to the current environment. If a particular classifier corresponds to the environment, it is added to the match set; otherwise it is ignored and the process continues until considering all members of the population. Every time there is a match, a counter is incremented to keep track of the match set size. If the match set is smaller than a specified threshold value, after comparing all classifiers from the population, a “covering procedure” is invoked as described below and shown in Fig. 2.
Fig. 2. Flowchart of matching and covering procedures
The covering procedure is used to guarantee that there was a sufficient number of classifiers that matched the environment. Therefore, when covering is required, classifiers are created in such way that conditions are equal to the environment, and actions are generated using pseudorandom numbers. Classifiers generated by covering are added to the match set and to the current population. If the population is considered to be too large, the same number of classifiers generated by covering are removed from the population, deleting the classifiers with the lowest fitness.
System Prediction and Action Selection
The prediction estimates the payoff that the system expects if a given classifier is used to take an action on a gate. At every time step, the XCS creates a new matching set, , which contains the classifiers that match the current environment. For each action in the match set, the XCS computed the system prediction , as shown in Fig. 3. The system prediction is computed as a weighted average of the predictions based on classifier fitness , as shown in Eq. (1)
(1)
where of classifier cl; of classifier cl; and of classifier cl.
Fig. 3. Flowchart of the prediction array generation procedure
The summations in Eq. (1) are performed over all classifiers that invoked action “,” and which belong to match set . The different values of are collected in the prediction array, which is the basis for action selection. In each case, the XCS selects the action with the greatest system prediction from the values in the prediction array. Then, all classifiers in the match set which contained this action are added to the action set.
Parameter Updates
During the learning process, the prediction , the prediction error , and the fitness are periodically updated. As a result, fitness is computed for every rule in the action set. Initial values for prediction were very close to zero for the initial population. The prediction , which gives an estimate of the payoff that the system is expected to gain when the classifier is selected, is updated as follows:
(2)
where rate and reward, which is computed as
(3)
in which reward; factor ; and maximum system prediction for this action.
The prediction error , which estimates the precision of the classifier’s prediction , is updated as follows:
(4)
To update the classifier fitness, the accuracy is determined as follows:
(5)
otherwise
(6)
If the prediction error is below the threshold error , the classifier is said to be accurate, meaning that has an accuracy of one, and errors are regarded as having equal accuracy; otherwise, the accuracy drops off quickly, depending on the values of and , which are parameters used to define the exponential function during model calibration.
The accuracy values for the classifiers in the action set are converted to relative accuracies as follows:
(7)
The relative accuracy is used to compute the classifier fitness , which is an estimate of the classifier’s accuracy relative to other classifiers in the action set . The fitness evaluates the accuracy of the prediction. Classifier fitness is updated toward the classifier’s current relative accuracy as
(8)
Genetic Algorithm
The GA selects two classifiers with probabilities proportional to their fitness. These classifiers act as parents to generate two children. Initially, one child was identical to one of the parents, and the second child was the same as the second parent. Subsequently, crossover and or mutation could take place in the children with specified probabilities and , respectively, as shown in Fig. 4. Two-point crossover is applied and one random bit is mutated whenever crossover and or mutation are invoked. The children are inserted into the population to compete with their parents and all other members of the population. The prediction and the prediction error for child rules are the averages of the two parents. The fitness was set as 10% of the average parent fitness to avoid bias toward a parent’s strength, thereby forcing children to compete as new members in the population.
Fig. 4. Flowchart of the genetic algorithm for introducing children
A “numerosity” value was defined as the number of identical (redundant) classifiers in a population to reduce the database size by manipulating the magnitude of this parameter while deleting repetitions. The child numerosity, which is the number of identical classifiers in the population, was given a value of one. The children’s experience value, which is the number of times that the child participated as parent in a genetic algorithm, was set to zero. When a member of the current population is found to be identical to a generated child rule, the classifier’s numerosity for the existing member is incremented by one, and the child is not added to the population.
In this study, different initial populations were tested. The first versions contemplated huge populations containing millions of members which were stored in a relational database to accommodate the size. The XCS main parameters were not randomly selected; instead, default values were assigned for the initial population. The approach of starting with a huge population had two disadvantages. First, it was time consuming, because all members were generated bit-by-bit using pseudorandom numbers, and second because database input and output consumed a large amount of computer processing time. After trying different sizes smaller than the initial approach, with thousands of members, and without success, a tiny population of 10 members was selected as an initial population. The advantage of this approach was that a small population could be defined with relatively strong rules, its generation took less than one second (instead of hours), and database processing was more streamlined. The presence of “good” rules in the initial population was found to be irrelevant, because the population quickly grew in size, and the XCS created rules with better fitness in the long run. Therefore, the final version for the initial population was one with only ten members, and it was generated randomly.
The population usually grew each time the GA was applied. Due to the small size of the initial population the deletion of classifiers from the population was disabled, thereby allowing the population to grow. Default values for the main XCS parameters were selected according to Butz and Wilson (2000), which established that prediction, prediction error, and fitness should have very small values. Thus, the default value used for prediction was 0.05; the prediction error was 0.001; and, the fitness was 0.001. Additional simulations were performed for model calibration, and multiple default values were tested, showing imperceptible changes in the results because the default values are for an initial population that changes as the optimization procedure progresses, due to the consistent parameter updating that takes place during the evolutionary process as described above.
Local and Global Hydraulic Variables
The variables of interest to run the program were defined by taking into account that they had to satisfy operational objectives. Local hydraulic variables were those that were taken in close proximity to a given gate in the canal system, and global variables were based on systemwide conditions. A strong emphasis was given to the balance of local and global hydraulic variables and their impact on the classifier system as a whole. Operational actions were influenced by the magnitudes of hydraulic variables and they were applied locally as well as globally, depending on how beneficial they were for individual canal reaches, and for the entire system. Hydraulic indicators governed the actions to be applied to the canal system and included the following local and global variables.
Local Variables
•
Deviation from the target depth for the previous and current simulation time steps of the hydraulic model;
•
A local hydraulic stability index (SI);
•
Current gate structure setting;
•
Average velocity change between hydraulic model simulation time steps in a reach; and
•
Infeasible operations.
Global Variables
•
Supply flow deviation from required amounts of water delivery at turnouts;
•
A system hydraulic SI;
•
Average velocity change between consecutive canal reaches;
•
Average velocity change between hydraulic model simulation time steps in a reach; and
•
Average discharge change at gate structures throughout the system.
The hydraulic SI is a measure of the hydraulic stability of the system layout as defined at the time of a simulation and it is computed as defined in the hydraulic model (Merkley 2007) as
(9)
where index; ; and . When computed for one particular pool (or reach), it was considered as a local SI and when computed using total inflows and outflows for the whole network of canals it was considered as a system SI. The index is equal to zero when the simulation is at, or is passing through, a stable hydraulic condition, meaning that all water depths and flow rates are unchanging with time (even though the flow rate and depth are likely to be changing as a function of distance along the reaches). When the index is a positive nonzero value, there is more flow entering the system than leaving, and vice versa for negative values. The index can vary from a minimum of −1.0 to a maximum of .
Average velocities were computed along each reach as the average of the mean velocity at the upstream end of the reach and the downstream end of the same reach. The mean velocity was computed as , where rate and area of flow. Therefore, the average velocity change between hydraulic model simulation time steps in a reach was computed as the algebraic difference between the average velocity in a reach in a particular time step and the correspondent average velocity for the previous time step. The average velocity change between consecutive canal reaches was computed as the average for all velocity changes for the whole hydraulic system.
Condition String
The condition string was the concatenation of substrings which defined the magnitude of a particular variable (local or global). Every substring was a concatenation of binary bits. A substring defined an instance value that a variable represented. The condition string contained variables that described the hydraulic state of the canal system. However, a given condition string might or might not describe a physically feasible situation because each string was randomly generated. For the purpose of constructing the condition string some local and some global variables were used, as follows:
•
Supply flow deviation from total water demand;
•
System hydraulic SI;
•
Average velocity change between consecutive reaches;
•
Current depth deviation from the target; and
•
Current structure setting.
The current depth deviation and gate setting were computed for every “pivot point” and every gate, respectively. The pivot point was the location in a reach where a target depth was specified, and was always either the upstream or downstream end of a reach. This implied that canal systems with more pools had larger condition strings than systems with less pools, because they have more gates and pivot points. These two parameters were included in the condition string as local variables, but they behaved as a single global variable. The current hydraulic state for the canal system, known as the environment string, had the same format as a condition string, but it represented a particular situation which was currently occurring in the canal system at that time step. This current state was translated from the hydraulic model results to a string, which was subsequently matched to one or more condition strings in the population.
Substrings were the supply flow deviation from delivery demands, the system SI, the average velocity change between consecutive reaches, the water depth deviation, and the current gate setting for the time step in consideration. The substring representing the current gate setting was located along the condition string immediately after the depth deviation partial string. These two substrings were repeated as many times as necessary in the canal system, but their magnitudes were different, in general.
Action String
The action string was a concatenation of actions taken for all gates in the canal system. Each part of this concatenation contemplated the option of opening or closing each gate by a specific amount. Therefore, the length of the action string depended upon the number of gates in the system. The virtue of this concatenation was that it combined all local actions into a single string, and at the same time it had a global context as it was a complete set for the entire system under consideration. Global actions generated widespread structure operations involving one or several structures along the canal system. Local actions generated a change in structure settings which were closer to the place in reference.
Local actions produced an effect upstream and downstream structures. Global actions were applied over the whole canal system. Thus, both global and local actions merged as a unique action for each structure in the system. The resultant action was applied over structures located throughout canal reaches between the upstream source of water and the downstream end of the last reach. Of course, in some cases a local or global action was “maintain the previous setting, and do nothing.” The final action was bounded depending on the current gate setting to disable possible actions which could go beyond physical limits.
Hydraulic Model
The RootCanal hydraulic model (Merkley 2007) was used to interact with the XCS as an environment that changed according to applied gate structure actions, as shown in Fig. 1. The hydraulic model predicted how the system responded, hydraulically, to gate setting actions generated by the XCS. The hydraulic model was condensed into a dynamic link library and was used to define the canal system, including reaches, structures, and other system features. The first interaction of the hydraulic model with the XCS was to compute the initial situation. This response from the hydraulic model was an input to the XCS after conversion to an environment string. Each time an action was selected by the XCS, it was passed to the hydraulic model, and based on the simulated response from the canal system, rewards and penalties were applied by the reinforcement program. The selected action was a set of particular operations which were applied to different gate structures in the canal system. With these new structure settings, the hydraulic model computed a new current system status, and as a result, the following variables were updated: water depths, average velocities, reach inflows and outflows, SIs, and structure settings. This new situation was evaluated by the reinforcement program, and it was used to analyze the classifier population’s evolution process.
Computed values were the basis for comparison for reward and penalty estimation by means of the reinforcement program. Depths were compared to target values and depth deviations were calculated. Velocities and discharges were compared to determine how fast the system status was changing, and how close it was to a stable hydraulic condition. Current turnout delivery demands were compared to previous demands as a measure of operational performance. Gate settings were used to detect the magnitude of changes and the occurrence of infeasible operations.
Reinforcement Program
Rewards and penalties were assigned depending upon rule matching to the environment. The magnitude of these rewards or penalties depended upon the impact of global and local variables on the canal system. The core of the “apportionment of credit” (reward and penalty) algorithm was based on reaching and maintaining water depths at the respective target values, and on minimizing the demand-supply deviation and SI for the whole canal system. In addition, the velocity changes between consecutive reaches, and the velocity change in time for every reach through the canal system, had a strong influence over the apportionment of credit calculation.
When assigning rewards, it was important to keep the objective function in mind. The main role of the objective function was to minimize water depth fluctuations generated by gate operations, while maintaining the water depths at or near the respective targets. The objective function was also designed to minimize the hydraulic stabilization time after the completion of a series of demand hydrograph changes. Rewards were based on both global and local system responses to a particular gate setting action (opening or closing). Some of the flowcharts are presented below for illustration purposes, but due to space limitations not all are presented in this paper.
Local Rewards
Local rewards were applied per structure depending upon the variable used in the analysis, and the location in the canal system. Local rewards were applied to gates to reinforce local behavior, as global rewards did for the entire canal system.
Depth Deviation from the Target
After running the hydraulic model following a given gate structure setting change, the model recomputed water depths in each reach. These water depths were compared with the target depths, and a deviation was computed to show how far the current water depth was from the target depth. Whenever the current depth was at the target depth in a given reach, a full reward was applied for the corresponding action (see Fig. 5).
Fig. 5. Flowchart of reward assignment for depth deviation from the target level
Deviations from the target depth were positive when current depth was above the target, and negative otherwise. When the last deviation was compared with the previous deviation, a response from the system was analyzed, and local rewards assigned. When the current water depth was moving toward the target after a change in gate structure setting, a partial reward was applied proportionally, taking 100% for the last deviation as a basis for computation. In the case of target depth overshooting (changing the sign of the deviation from positive to negative, or vice versa), the reward was computed as mentioned above, but it was reduced according to the percentage of overshoot. For the case where the previous depth was closer to the target depth than the current depth, a penalty was assigned instead of a reward. Whenever a previous depth deviation was equal to the current deviation, no reward was applied, irrespective of how far the current depth was from the target.
A mechanism was developed to maintain water depths constant after they have reached the respective target values. For every reach, a “flag” variable, , was computed for the cases when the current depth was equal to the target depth. The flag variable was incremented by one each time the current depth remained at the target depth, and it was set equal to zero when the water depth did not remain at the target depth since the last time step. When the calculated depth was maintained at the target depth a higher reward (a bonus) was assigned. To avoid excessively large rewards with respect to other parameters, the reward for reaching the target depth was assigned an upper limit.
When the water depth reached the target the first time, it was rewarded by a basic amount, , and if it was maintained at the target depth, the reward grew exponentially as follows:
(10)
where for reaching the target depth; reward; ; of consecutive time steps remaining at the target; and .
Local SI
The reward from the SI was a function of the current and target water depths. A zero stability index was rewarded in full if the current water depth reached the target depth. Whenever the current depth was below the target depth, a positive SI was desirable because it increased the amount of water coming into the canal system at the source, leading to an increase in the depths, in general. If the current depth was above the target depth, decreasing the amount of water entering the system was a way to deplete water depths, and in this case a negative SI was rewarded, as seen in Fig. 6.
Fig. 6. Flowchart of reward assignment based on the local SI
Demand-supply deviation at turnouts was a factor that influenced the SI. An extra reward was assigned for the case when demand was equal to the supply, having a SI of zero, as well as having the current water depth at the target depth. This extra reward was because this is the most desirable situation when considered on a local basis.
To assign a reward for local SI, , the absolute value of the SI, , was used. The absolute value of the variable was required, because the reward should be the same for the case when outflows were greater than inflows, and vice versa. As the system approached a hydraulically stable condition, the SI approached zero. The SI had a range from −1.0 to . The reward for local SI was inversely proportional to the absolute value of the SI. As the SI reached zero when inflows were equal to outflows, the local SI reward, , reached its maximum value of . To compute the local SI reward, the absolute value of the SI, was affected by a coefficient, , and an exponent m which was used to weight it with respect to other rewards, as
(11)
Reach Velocity Change between XCS Time Steps
Reach velocity change between XCS time steps represented how fast the water was flowing through the system according to the magnitude of the last action applied to the canal system. In order to assign rewards for reach velocity change between XCS time steps, current and target depth were considered. Rewards were applied for velocity changes that reduce the difference between the current depth and the target. Whenever the current depth was below the target depth, a positive reach velocity change between simulations was rewarded. A null reach velocity change between simulations generated a reward, if the current depth was at the target depth. When the target depth was below the current depth, a negative reach velocity change leaded to a decrease in the depth, and this situation was rewarded.
Global Rewards
Global rewards had a major impact on the environment because they involved one, many, or all gate structures in the canal system. These rewards were applied spatially over all of the structures in the canal system, and they were based on the criteria described below.
Demand-Supply Deviation
System demand and supply were computed by the hydraulic model after an action was applied. Whenever demand equaled supply, a maximum reward was granted. If there was a deviation between demand and supply, proportional rewards were assigned according to the deviation. Demand-supply deviation values greater than a threshold value did not generate any reward. This applied to the apportionment of credit in general, so that if the absolute turnout demand-supply deviation was greater than a specified threshold value for the deviation, no reward was applied.
Global SI
To compute the global SI, total inflow and outflow from the system were considered. Reward from global SI was also a function of demand-supply deviation relationship. A zero SI was rewarded in full if demand was equal to supply, otherwise a partial reward was applied. Partial rewards were applied depending on the sign of the SI. Whenever the demand was greater than the supply flow rate, a positive SI was desired because it helped compensate for the volumetric deficit. Thus, a positive SI generated a partial reward when the demand was greater than the supply of water to the system. On the other hand, if supply was greater than demand, a negative SI was partially rewarded. The most desirable situation considering the entire system was the case when the current water depths were within a dead band, and the SI was zero. An extra reward is granted whenever these goals were met.
Change in Gate Discharge between XCS Time Steps
The magnitude of the change in discharge through gate structures between XCS time steps was indicative of the impact of the last applied set of actions over the whole system. Higher magnitudes of change meant that the last set of actions was more drastic relative to the current set of actions. A zero value meant that there was no net discharge change after the last action was applied. Rewards for change of discharge through gates between XCS time steps were applied according to the magnitude of the demand-supply deviation. Greater demand-supply deviations generated larger rewards for larger discharge changes than for smaller changes.
Velocity Change between Reaches
This parameter indicated how fast the water was moving through the system and, consequently, how fast the system responded Full reward was applied for zero velocity change between reaches when demand equaled supply. For positive demand-supply deviations, meaning demand greater than supply, rewards were applied for positive changes in velocity between reaches. For a case when supply was greater than demand, rewards were applied for decreasing velocities.
Therefore, applying the superposition principle to all mentioned rewards, the total reward was defined as the summation of rewards that apply to a particular gate structure. A gate structure had different sources of rewards depending on its particular condition. It had rewards granted locally, globally, or both at the same time.
Penalties
Penalties were assessed as a function of the number of consecutive times the water depth had not reached the target depth and the absolute value of the SI. Infeasible structure adjustments were also penalized.
Penalty for Leaving the Target Depth
A large penalty was assessed when a given rule caused the calculated depth to deviate from the target depth after it had been reached in the previous time step, and this penalty was increased if it was not achieved consecutively. After reaching the target depth, any deviation from the target depth was penalized. In this case, the penalty was computed as
(12)
where ; penalty; of times missing the target depth since last time it was at the target; and exponent, which is less than 1.0. The depth was considered to be correct whenever the calculated value was within a ±8% dead band around the target depth.
Penalty for Instability
An instability penalty, , was apportioned when the system had reached a hydraulically stable conditions, and the stability was lost during the next simulation. For large differences in the SI between two XCS time steps, greater penalties were applied. Low penalties were considered when the SI difference value was close to zero and maximum penalties when it was close to 1.0. The penalty was proportional to aforementioned difference, and it was adjusted according a coefficient, , and a specified exponent, u, to make the penalty proportional to other penalties
(13)
Penalty for Infeasible Operation
A structure operation penalty was computed when a rule corresponded to an infeasible operational adjustment. These were the cases when a structure with a current small opening was required to be closed by an amount greater than what was physical permissible, when a completely open structure was required to be opened an additional amount, or when a specified gate structure operation forced a flow regime change (e.g., orifice to nonorifice flow) at the structure.
Summary
An accuracy-based learning classifier system (XCS) and a multiobjective GA were developed using an unsteady-flow hydraulic model to simulate operational conditions in irrigation canals. The model was defined to answer the question of generating operational rules that maintain water depths inside a dead band, surrounding a target depth, with acceptable stability. The XCS was developed to produce acceptable operational solutions for gate structures in canal systems based on a specified objective function. A multiobjective function was defined to minimize: (1) water depth fluctuations; (2) the absolute value of the SI; and, (3) the demand-supply flow rate difference. The resulting set of operational rules obtained from the XCS can be applied in the field by deploying them in a data logger, which will require the development of code to communicate the recommended gate structure adjustments, according to the current state of the canal system.
Notation
The following symbols are used in this paper:
=
number of times missing the target depth;
=
coefficient for water depth deviation from the target;
=
coefficient for stability penalty;
=
number of consecutive repetitions when reaching the target depth;
=
coefficient for stability reward;
=
classifier fitness;
=
classifier prediction;
=
classifier action;
=
calibration exponent for penalty for leaving the target depth;
=
exponent for stability rewards;
=
exponent for water-level deviation from the target depth;
=
base penalty;
=
stability penalty;
=
penalty for leaving the target depth;
=
system prediction;
=
classifier’s maximum prediction for an action;
=
desired reward;
=
reward for stable;
=
base reward;
=
reward for reaching the target depth;
=
previous reward;
=
exponent for stability penalty;
=
parameter for accuracy;
=
learning rate;
=
discount factor;
=
prediction error;
=
prediction error threshold;
=
accuracy values for classifiers;
=
relative accuracy;
=
probability of mutation;
=
exponent for accuracy;
=
probability of crossover;
=
match set; and
=
absolute value of the SI.
Acknowledgments
Support from the Utah Agricultural Experiment Station is gratefully acknowledged.
References
Babayan, A. V., Kapelan, Z. S., Savic, D. A., and Walters, G. A. (2006). “Comparison of two methods for the stochastic least cost design of water distribution systems.” Eng. Optimiz., 38(3), 281–297.
Bautista, E., Strelkoff, T. S., and Clemmens, A. J. (2003). “General characteristics of solutions to the open-channel flow, feed-forward control problem.” J. Irrig. Drain. Eng., 129(2), 129–137.
Bernado-Mansilla, E., and Ho, T. K. (2005). “Domain of competence of XCS classifier system in complexity measurement space.” IEEE Trans. Evol. Comput., 9(1), 82–104.
Burt, C. M., Angold, R., and Lehmkuhl, M. (2001). “Flap gate design for automatic upstream canal water level control.” J. Irrig. Drain. Eng., 127(2), 84–91.
Burt, C. M., Mills, R. S., and Khalsa, R. D. (1998). “Improved proportional-integral (PI) logic for canal automation.” J. Irrig. Drain. Eng., 124(1), 53–57.
Butz, M. V., Kovacs, T., Lanzi, P. L., and Wilson, S. W. (2004). “Toward a theory of generalization and learning in XCS.” IEEE Trans. Evol. Comput., 8(1), 28–46.
Butz, M. V., and Wilson, S. W. (2000). “An algorithmic description of XCS.” Proc., Advances in Learning Classifier Systems: Proceedings of the 3rd Int. Workshop, P. Lanzi, W. Stolzmann, and S. Wilson, eds., Springer, Berlin, Heidelberg, Germany, 253–272.
Buyalski, C. P., Ehler, D. G., Falvey, H. T., Rogers, D. C., and Serfozo, E. A. (1991). Canal systems automation manual, Vol. 1, U.S. Bureau of Reclamation, Denver.
Chittaladakorn, S., and Merkley, G. P. (2005). “Classifier system for rule-based operation of canal gates.” J. Water Resour. Plann. Manage., 131(1), 3–13.
Clemmens, A. J., Wang, C., and Replogle, J. A. (1990). “Dynamic response of automatic water-level controller: Application.” J. Irrig. Drain. Eng., 116(6), 784–796.
Dandy, G. C., and Engelhardt, M. (2001). “Optimal scheduling of water pipe replacement using genetic algorithms.” J. Water Resour. Plann. Manage., 127(4), 214–223.
Jain, A., Bhattacharjya, R. K., and Sanaga, S. (2004). “Optimal design of composite channels using genetic algorithm.” J. Irrig. Drain. Eng., 130(4), 286–295.
Kuo, S. F., Merkley, G. P., and Liu, C. W. (2000). “Decision support for irrigation project planning using a genetic algorithm.” Agric. Water Manage., 45(3), 243–266.
Litrico, X., Malaterre, P. O., Baume, J. P., and Ribot-Bruno, J. (2007). “Automatic tuning of PI controllers for an irrigation canal pool.” J. Irrig. Drain. Eng., 133(1), 27–37.
Montesinos, P., Camacho, E., and Alvarez, S. (2002). “Application of genetic algorithms for optimal seasonal furrow irrigation.” J. Hydroinform., 4(3), 145–156.
Nagesh Kumar, D., Srinivasa, K., and Ashok, B. (2006). “Optimal reservoir operation for irrigation of multiple crops using genetic algorithms.” J. Irrig. Drain. Eng., 132(2), 123–129.
Nixon, J. B., Dandy, G. C., and Simpson, A. R. (2001). “A genetic algorithm for optimizing off-farm irrigation scheduling.” J. Hydroinform., 3(1), 11–22.
Peng, W., and Jia, R. (2004). “Improved genetic algorithms for optimal design of drainage systems.” Proc., 8th Int. Conf. in Control, Automation, Robotics, and Vision, Nanyang Technical University, Singapore, 227–231.
Pezzinga, G., Gueli, R., Reis, L. F., Porto, R. M., and Chaudhry, F. H. (1999). “Optimal location of control valves in pipe networks by genetic algorithm. Discussion.” J. Water Resour. Plann. Manage., 125(1), 65–66.
Reddy, J. M., Dia, A., and Oussou, A. (1992). “Design of control algorithm for operation of irrigation canals.” J. Irrig. Drain. Eng., 118(6), 852–867.
Schuurmans, J., Clemmens, A. J., and Dijkstra, S. (1999a). “Modeling of irrigation and drainage canals for controller design.” J. Irrig. Drain. Eng., 125(6), 338–344.
Schuurmans, J., Hof, A., and Dijkstra, S. (1999b). “Simple water level controller for irrigation and drainage.” J. Irrig. Drain. Eng., 125(4), 189–195.
Simpson, A. R., and Wu, Z. Y. (2001). “Competent genetic-evolutionary optimization of water distribution systems.” J. Comput. Civ. Eng., 15(2), 89–101.
Stringam, B. L. (1998). “A fuzzy logic controller for downstream water-level control in canals.” Ph.D. dissertation, Dept. of Bio. and Irrig. Engineering, Utah State Univ., Logan, Utah.
Zheng, Y. W. (2006). “Optimal calibration method for water distribution water quality model.” J. Environ. Sci. Health, Part A: Toxic/Hazard. Subst. Environ. Eng., 41, 1363–1378.
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
IR.1943-4774.0000268/asset/9782fe3c-c00c-47b6-a08b-06afafeef5eb/assets/images/large/1.jpg)
IR.1943-4774.0000268/asset/468ea342-87b5-4574-8a1d-1779892f3a4f/assets/images/large/2.jpg)
IR.1943-4774.0000268/asset/9dcab185-4e27-45fc-8f08-868a7f3a93f5/assets/images/large/3.jpg)
IR.1943-4774.0000268/asset/fbd6faa2-c824-4250-a229-918afefaa30c/assets/images/large/4.jpg)
IR.1943-4774.0000268/asset/7feb3bd1-ad51-45f2-ac18-a7304bcf4a2b/assets/images/large/5.jpg)
IR.1943-4774.0000268/asset/87682634-0c90-479b-81e3-f3e5a3f0c335/assets/images/large/6.jpg)
