Chapter 3 Synthesis of analogue circuits 1

3.3.8 Automatic design and the use of Artificial Intelligence

Automatic design of analogue circuits has been theoretically possible for some time now, using the algorithms that we have studied in the previous sections for the analysis of such circuits. It should be possible to design a circuit to meet given requirements by the repeated application of an automatic circuit analysis programme (such as Spice) and a suitably defined performance criterion, merely by closing the loop. However, it has been found that the computational burden of such an algorithm is unbearably high, and that it is not possible to obtain a satisfactory solution within reasonable limits of time and effort. An alternative approach using genetic algorithms has been suggested recently, and satisfactory results have been reported for the design of analogue circuits, notably for the design of filter circuits. In fact, designs superior to those obtained by experienced design engineers using conventional design techniques have been reported to have been obtained using these techniques. There are some variations among the methods reported in the literature and we will discuss some of them below. Genetic algorithms Generic algorithms refer to a class of search techniques that try to emulate the natural phenomenon of evolution. It was first proposed by Holland in the 1960s as a means for the study of the phenomenon of adaptation in nature. He also intended to import the mechanisms of natural adaptation to computer systems. However, our intention here is to use it for the solution of a specific problem, that of searching for an optimal or semi-optimal solution to a stated problem in circuit design. Genetic algorithms are particularly suited for the solution of problems where the search domain is very large. This is the case with analogue circuit design, for even with self-imposed limitations such as the use of resistors, capacitors and inductors only as circuit elements, and on the number of components to be used, the combinations of connectivity patterns as well as of parameter values is almost infinite. Chromosomes and genes Borrowing concepts and vocabulary from evolutionary genetics, each potential solution is described by a chromosome, consisting of a number of genes. Traditionally, a chromosome consists of a fixed number of genes, and the genes themselves are coded in binary form, even though the literature contains instances where variable length chromosomes and non-binary coded genes have been used.

2 A systems approach to circuits, measurements and control

Population and fitness A population, again by analogy with evolutionary biology, consists of a number of chromosomes representing different individuals or different possible solutions. Each member of the population will be associated with a fitness value, corresponding to how well it meets the desired requirements. Parents, children and succeeding generations The GA (genetic algorithm) works through the creation of successive generations, each fitter than its predecessor, by a process of constructing children (of a new generation) from parents (of the older generation). This is accomplished through a process of selection, crossover and mutation. The complete process is illustrated in the following chart.
Start with a randomly generated population of n l-bit chromosomes.

Repeat for required number of iterations (or generations)

Calculate the fitness f(x) of each chromosome x

Repeat until n chromosomes (a new generation) are created

Select a pair of chromosomes (the probability of being chosen is proportional to fitness) to be parents of a new chromosome

Form a pair of children from the pair of parent chromosomes. The children are formed either by crossover (with specified crossover probability) or, if no crossover occurs, by replication of the parents.

Subject the two new chromosomes to mutation, with specified probability of mutation.

Place the children in a new generation of chromosomes

One Generation

Select the fittest chromosome as the required solution

Stop

Chapter 3 Synthesis of analogue circuits 3

The above is a generic genetic algorithm that could be used for the solution of problems in any domain. We need to make a number of decisions if we are to apply it to the solution of a particular problem, in a particular domain. The first, and perhaps the most difficult, is how to present the problem in a suitable manner. Two different approaches to passive analogue circuit design using genetic algorithms have been reported, both with some measure of success. In one approach, both the topology of the network as well as its parameter values (the values of the resistors, capacitors and inductors) have been optimised together in one process, through a suitable coding mechanism. In the other, only the topology is represented in the GA, and the optimal parameter values for each configuration is obtained using conventional optimisation methods. [This seems a better idea, for the optimisation of parameter values can be handled more economically by conventional methods.] These two processes are repeated until a satisfactory solution is obtained. There are many other variations in these algorithms, and different workers have reported different procedures. For example, in some algorithms, a chromosome from a child generation will only replace a member of the parent generation having a lower fitness than itself, while in others there is no such check. The other major decision is about the fitness function. The fitness function should represent how well the individual (chromosome) meets the design requirements. These may be specified either in the frequency domain or in the time domain, or may even be a combination of the two. As it is possible to meet tighter specifications with higher order circuits with more components, there will be a tendency to increase the number of components indefinitely to achieve a better fit. This is of course counter productive, as both size and costs will escalate with increasing number of components. The fitness function should be designed to take this into account by punishing designs using large numbers of components. In a typical implementation, the fitness function has been multiplied by a penalty function p(n) of the number of components n, where it has been defined as: 1 p ( n) = ( n n max) 1+ a The figure shows plots of p(n) for a = 10 for nmax = 5 and 10. It illustrates how the penalty function decreases rapidly as n passes nmax.

4 A systems approach to circuits, measurements and control

An example Before we consider the problem of analogue circuit design with all its intricacies, we will consider a simpler example to try to understand how a genetic algorithm works. Let us consider the problem of maximising the function

sin x f ( x) = abs 2 1 + x
over the range 16 < x < 16 We will first try to gain an insight into the problem by examining how this function behaves over the range of interest. The figure shows a plot of f(x) verses x.

Note that f(x) is a continuous function, and that we need to divide the search space into fairly small segments. As we are interested in representing each solution (that is, each value of x) as a chromosome made up of binary valued genes, one possibility is to construct a chromosome consisting of (say) ten genes, so that we divide the range (-16,16) into 1024 segments. If we consider the number represented by the ten bits as j, then x would be (j-512)/32, for 0 < j 1024 . We will attempt to implement a genetic algorithm to solve this problem using MATLAB. 1. Define the function to be optimised (maximised): To do this we first type edit in the MATLAB command window. This opens a new window for editing. In the window we can create an m file defining f(x) as follows:
function[y]=f(x) y=abs(sin(x)/(1.+x^2));

This is then saved as the m file f.m. Let us try this out (as we already have a plot of the function) by substituting different values for x:

Chapter 3 Synthesis of analogue circuits 5

>> f(-5.) ans = 0.0369 >> f(0) ans = 0 >> f(1) ans = 0.4207

2. We will now create a (random) population (say) of 100 to start the algorithm. As each solution is represented by a binary string of 10 bits (corresponding to 1024), the initial population may be represented by a 100 x 10 random matrix of ones and zeros. In more general terms, the stating population corresponds to a (n x m) matrix, where n is the population size and m is the length of each chromosome. We will again invoke edit from the MATLAB command window and create a new file as follows: function[y]=starting_population(n,m) y=round(rand(n,m)); The MATLAB command rand generates a matrix (of given size) of random numbers in the range (0,1.0) while round rounds off a given number to the nearest integer. Together, they generate a matrix of random ones and zeros as required. Let us try it out (for a starting population of 10, to save space!) :
>> sp=starting_population(10,10) sp = 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 0 1 0 0 0 1 0 1 1 1 1 0 0 0 1 0 1 1 0 1 0 0 1 1 0 1 1 0 0 1 1 0 1 0 0 0 0 0 0 1 0 0 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 0 1 0 0 1 1 0 0 1 0 1 1 0 0 0 0 1 0 1 0 0 1

6 A systems approach to circuits, measurements and control

Each row corresponds to one individual (one chromosome) of the starting population [Note that re-running the programme will generate a different set of values.] We may write an m file to extract any chromosome we wish from the total population:
function[y]=chromosome(population,j) [n,m]=size (population); y=zeros(m); y=population(j,:);

We will try this out for a few chromosomes:

> chromosome(sp,1) ans = 1 0 0 0 1 0 1 0 0 0

>> chromosome(sp,5) ans = 0 0 1 1 0 0 1 0 0 1

. In terms of decimal values (in the range 0 1023), our starting population correspond to: 552, 64, 930, 354, 201, 388, 1015, 684, 778, 639

Page 3

chromosome, and replace it with a new gene of random type and random terminal nodes. Unfortunately this process generates a high proportion of lethal mutations in which a viable circuit is transformed into a non-viable circuit. The success rate of mutation can be

Chapter 3 Synthesis of analogue circuits 7

improved by modifying the circuit only in ways that are likely (but not guaranteed) to lead to a viable result. One of the following four circuit transformations, selected at random, is applied to the offspring's chromosome: 1. Replace an existing component by an open circuit. 2. Replace an existing component by a short circuit. 3. Connect a new random component in parallel with an existing component. 4. Connect a new random component in series with an existing component. These operations alone would be sufficient to transform any given circuit into any other circuit, and it might be thought that cross-over is therefore superfluous. This is in fact not the case as the use of cross-over greatly improves the efficiency. Once a circuit topology has been generated its fitness is evaluated after numerically optimizing its component values using a quasi-Newton algorithm based on the Davidon-FletcherPowell (DFP) method. This calls an objective function that returns the sum-of-squares of the differences between the circuit's response and the target response at a sequence of logarithmically-spaced frequencies. To reduce the amount of computation involved a symbolic analysis of each new circuit topology is performed before numerical optimization. This involves constructing voltage and current graphs (corresponding to the voltage and current incidence matrices) from the circuit, and after coalescing appropriate pairs of nodes, finding all the common spanning trees of the two graphs [7]. The result is two linked lists of symbolic terms, corresponding to the

8 A systems approach to circuits, measurements and control

numerator and denominator of the voltage frequency-response function. Substituting component values into the symbolic form gives the numerical frequency-response function; substituting frequency into the numerical frequency-response function gives the voltage gain. As the response approaches the target response, the objective function tends to zero and the reciprocal of the objective function for the optimized circuit is returned as the fitness. In the absence of any other constraints the GA will generate successively more complex circuits, because a complex circuit will in general provide a better fit to a target response than a simple circuit. To prevent this the fitness is multiplied by a penalty function p(n) that is unity for simple circuits, but which rapidly becomes smaller as the complexity (measured by the number of nodes n) exceeds some predetermined level n_max: max) _ ( 1 1)(nna np+ = 1. where a is a constant, typically 8 and n_max is set to the anticipated complexity. This hybrid-GA circuit synthesis method is remarkable for incorporating no design rules or expert knowledge; it simply works towards satisfying the design goals. It is the antithesis of the traditional "expert system" approach to analogue circuit design. 5 A Simple Frequency-Domain Filter Benchmark An obvious way of testing the effectiveness of the hybrid-GA is to synthesize a circuit to a specification for which a formal design method exists. Consider the normalized low-pass filter

Chapter 3 Synthesis of analogue circuits 9

specification: Pass-band edge: 1.0 rad/s Stop-band edge: 1.5 rad/s Maximum pass-band gain: 0 dB Minimum pass-band gain: -1 dB Maximum stop-band gain: -46 dB Following the traditional filter design procedure, the first stage is to choose a suitable filter approximation. Provided that pass-band
ripples and a non-monotonic stop-band are Page 3

chromosome, and replace it with a new gene of random type and random terminal nodes. Unfortunately this process generates a high proportion of lethal mutations in which a viable circuit is transformed into a non-viable circuit. The success rate of mutation can be improved by modifying the circuit only in ways that are likely (but not guaranteed) to lead to a viable result. One of the following four circuit transformations, selected at random, is applied to the offspring's chromosome: 1. Replace an existing component by an open circuit. 2. Replace an existing component by a short circuit. 3. Connect a new random component in parallel with an existing component. 4. Connect a new random component in series with an existing component. These operations alone would be sufficient to transform any given circuit into any other circuit, and it might be thought that cross-over is therefore superfluous. This is in fact not the case as the use of cross-over greatly improves the efficiency. Once a circuit topology has been generated its

10 A systems approach to circuits, measurements and control

fitness is evaluated after numerically optimizing its component values using a quasi-Newton algorithm based on the Davidon-FletcherPowell (DFP) method. This calls an objective function that returns the sum-of-squares of the differences between the circuit's response and the target response at a sequence of logarithmically-spaced frequencies. To reduce the amount of computation involved a symbolic analysis of each new circuit topology is performed before numerical optimization. This involves constructing voltage and current graphs (corresponding to the voltage and current incidence matrices) from the circuit, and after coalescing appropriate pairs of nodes, finding all the common spanning trees of the two graphs [7]. The result is two linked lists of symbolic terms, corresponding to the numerator and denominator of the voltage frequency-response function. Substituting component values into the symbolic form gives the numerical frequency-response function; substituting frequency into the numerical frequency-response function gives the voltage gain. As the response approaches the target response, the objective function tends to zero and the reciprocal of the objective function for the optimized circuit is returned as the fitness. In the absence of any other constraints the GA will generate successively more complex circuits, because a complex circuit will in general provide a better fit to a target response than a simple circuit. To prevent this the fitness is multiplied by a penalty function p(n) that is unity for simple circuits, but which rapidly becomes smaller as the complexity (measured by the number of nodes n) exceeds some predetermined level n_max:

Chapter 3 Synthesis of analogue circuits 11

max) _ ( 1 1)(nna np+

= 1. where a is a constant, typically 8 and n_max is set to the anticipated complexity. This hybrid-GA circuit synthesis method is remarkable for incorporating no design rules or expert knowledge; it simply works towards satisfying the design goals. It is the antithesis of the traditional "expert system" approach to analogue circuit design. 5 A Simple Frequency-Domain Filter Benchmark An obvious way of testing the effectiveness of the hybrid-GA is to synthesize a circuit to a specification for which a formal design method exists. Consider the normalized low-pass filter specification: Pass-band edge: 1.0 rad/s Stop-band edge: 1.5 rad/s Maximum pass-band gain: 0 dB Minimum pass-band gain: -1 dB Maximum stop-band gain: -46 dB Following the traditional filter design procedure, the first stage is to choose a suitable filter approximation. Provided that pass-band
ripples and a non-monotonic stop-band are