Optimal allocation of microgrid using a differential multi-agent multi-objective evolution algorithm

Abstract The optimal configuration and allocation of a microgrid are one of the key issues to guarantee the economic and reliable working of a microgrid. This is a multi-objective optimisation problem within which the economic index and the load power shortage rate index should be considered when optimising the configuration. In this article, a differential multi-agent multi-objective evolutionary algorithm (DMAMOEA) was designed to optimise the capacity configuration of a microgrid system, which includes three kinds of equipment: wind turbine, photovoltaic equipment and battery. The final optimisation results were compared with the Non-dominated Sorting Genetic Algorithm II (NSGA-II) algorithm. Simulation results showed the effectiveness of the algorithm. At the end of this article, the representative solutions in the calculation results are compared and explained and the environmental benefits are analysed, which show the effectiveness of the implementation of the microgrid system.


Introduction
Compared with traditional fossil energy, renewable and clean energy such as wind energy and solar energy are safe, pollution-free, widely distributed and conducive to small-scale decentralised utilisation [1]. With the growth of these clean and renewable energies, distributed generation has created a steadily increasing amount of research interest. To improve security, stability and power quality, it is an effective way to exert the efficiency of distributed generation system in the form of a microgrid [2].
A microgrid is an automatic and stand-alone system, which can realise self-control, safeguard and administration. From the macroscopic view, a microgrid can be seen as a 'virtual' power supply source or a load in the network. The optimal configuration of a microgrid is one of the key issues to ensure the economic and reliable operation of the microgrid. It is generally necessary to consider the following aspects: economic indicator, selfpower supply capacity indicator and environmental protection indicator in the microgrid allocation optimisation problem. The economic indicator mainly reflects the economy of microgrid operation, such as the construction of distributed generation, operation and maintenance costs, replacement costs and fuel costs [3]. The load power shortage rate indicator refers to the power supply performance of the microgrid, which reflects the ability of the microgrid to meet the load requirements of the microgrid in an independent state. Environmental protection indicator refers to the environmental benefits of microgrid operation. Through environmental protection indicators, the advantages of the environmental protection benefits of the microgrid can be measured. In addition, targets such as power loss minimisation and voltage profile improvement are also considered. Therefore, the optimal configuration of the microgrid is a multi-objective optimisation problem.
In the optimisation of capacity allocation, the economic requirement can be considered separately according to specific needs [4,5]. In this situation, the allocation problem is a single objective optimisation problem. However, in most cases, multiple objectives should be considered. When optimising multiple objective functions, multiple objectives can be integrated according to certain weights and then solved by the single-objective optimisation method [6]. A multi-objective optimisation algorithm based on Pareto optimal solution can also be used to solve the problem [7]. In this process, the Pareto solution set of the problem to be optimised can be obtained by a single run. Compared with the weight method, the latter can get the solution set more comprehensively and efficiently. In fact, the weight method takes only a small part in solving multi-objective optimisation problems, and the rest employ the Pareto front method.
In the process of solving the multi-objective capacity allocation problem, many algorithms have been designed and applied. Compared with the basic intelligent algorithm, a hybrid intelligent algorithm has many outstanding advantages in solution quality, problem processing scale and convergence speed, and therefore it may be more suitable for solving such problems [8][9][10]. Meanwhile, the utility of agents for solving problems has aroused much concern these years. The concept of agents and their generation, which constructs a multi-agent system (MAS) and allows different search spaces to be explored simultaneously, thereby achieving solutions with more diversity and high quality [11].
In view of the advantages of hybrid intelligent algorithms and the multi-agent approaches, approaches with multi-agent technology are a good choice in the process of hybridisation [9]. In this study, a differential multiagent multi-objective evolutionary algorithm (DMAMOEA) was designed by combining differential evolution strategy and multi-agent technology, which is used to solve the capacity optimisation problem of the microgrid including wind turbine, photovoltaic equipment and battery storage, considering the two objectives of economy and load outage rate.
Comparing the results with the classical multi-objective evolutionary algorithm -Non-dominated Sorting Genetic Algorithm II (NSGA-II), the quality of the final solution set and solution time of the designed algorithm is better than that of the classical algorithm. Subsequently, some representative solution in the solution set is analysed and the effectiveness of microgrid implementation is also illustrated.

Microgrid-related equipment model and operation mode
In this work, the optimal allocation model of microgrid capacity includes three kinds of equipment: photovoltaic power production facility, wind power production facility and storage battery. The output models of the three kinds of equipment are as follows.

Photovoltaic power generation model
The power output of the photovoltaic array can be described as follows as in Eq. (1) [12]: Where, f PV is the power reduction factor of the photovoltaic system, representing the ratio of actual output power to the rated output power of photovoltaic systems; the value of f PV is generally 0.9; Y PV is the capacity of the photovoltaic array; kW is the unit; I T is the actual illumination, and the unit is kW/m 2 ; T S is the illumination under the standard test conditions, generally take 1 kW/m 2 ; α denotes the power-temperature coefficient, the unit is %/ • C; T cell is the current temperature of the surface of the photovoltaic cell, which can be estimated according to the current environmental temperature; and T cell,STC is the photovoltaic cell temperature in the standard test environment, which is generally taken as 25 • C.

Wind turbine output model
The output power of a wind turbine fluctuates with the fluctuation of wind speed. Thus, the actual distribution of wind speed must be obtained to calculate the power output of the wind turbine. It can be pointed out that the wind speed distributed on the ground is different from the height of the measuring point. The real-time wind speed data provided by the Meteorological Bureau are generally measured near the altitude of 9 m. To get the actual output power of the wind turbine, the measured wind speed should be converted into the speed at the height of the rotating shaft of the wind turbine. The conversion can be roughly calculated by the following equation: [13,14].
Where, v and v 0 are the wind speeds in H and H 0 height, respectively; n is the correction index, which is related to the surface roughness and atmospheric stability. The value of n is normally between 1/2 and 1/8. When the wind speed and its distribution are known, the output power of the wind turbine can be obtained by the functional relationship between the output power and the wind speed. Its output function can be described as follows: Where P W T is the output power of the wind turbine; P W r is the rated output power of the wind turbine; V ci is the cut-in wind speed; V co is the cut-off wind speed; and V r is the rated wind speed.

Battery output model
The remaining electricity of the battery at time t has a bearing on the remaining electricity of the battery at time t-1, charge or discharge capacity of batteries during the [t-1, t] period and the self-discharge.
The residual electric quantity can be described as under when the battery discharges or charges [15].
Where, S (t) is the residual electric quantity; P SB (t) is the discharge or charge or discharge power; η C and η D are the charge or discharging efficiency, respectively; σ is the self-discharging ratio. The signature of P SB (t) is positive when the battery discharges, while it is negative when charges.

The operation mode of microgrid
The power outputted from wind turbines and solar photovoltaic power generation equipment is related to local weather conditions, which are random and cannot be adjusted artificially. Battery energy storage systems can be charged and discharged within a certain range and can supplement the difference between renewable energy (wind turbine and photovoltaic power generation) and load demand according to specific conditions. In addition, by connecting power lines, the microgrid can exchange energy with the main network, sell surplus power to the main network or purchase power from the main network to satisfy the load demand of the microgrid. In this work, the overall energy control strategy is to use the power of wind turbine and photovoltaic unit preferentially in the microgrid system, and the battery plays the role of energy buffer and system standby.
A year is divided into 8760 h. Assuming that the output power of wind turbine and photovoltaic remains constant within an hour, the difference between the system load demand and the total wind and photovoltaic power generation is calculated according to the mathematical model of each micro-source.
In this case, there are some operation principles with the microgrid. They are mentioned below: 1) If the power generated by renewable energy generation equipment equals the load demand of the microgrid, the batteries don't charge or discharge, and no energy interacts between the microgrid and main network.
2) If the power generated by the renewable energy equipment surplus the load demand in the microgrid, the batteries should be charged preferentially with the allowance state of the batteries.
3) If there is still surplus power, it will be sold to the external power grid as far as possible within the power limit of tie-line backward transmission, and the remaining power will be the system energy spillover. 4) If the net load is greater than zero, the renewable energy generation power is insufficient. Under the conditions of the batteries discharge power and the state are allowed, they are preferentially used to balance the load in a microgrid. When the power which is discharged by the batteries cannot meet the load demand, the left power is purchased from the main network to ensure power balance in the microgrid.

The objectives and constraints
In the actual micro-grid operation process, it is necessary to consider not only the economics of the microgrid but also power supply reliability and environmental protection. In this work, the economy and power supply reliability are considered in the multi-objective planning and design of the microgrid, which is a two-objective optimisation problem. The optimised independent variables are the number of photovoltaic power generation equipment N pv , the number of wind turbines N wind , and the number of energy storage batteries N battery .

The cost of net present value of life cycle
The cost of Net Present Value (N PV ) of life cycle expense mainly includes the following aspects: cost for initial investment, the cost for operation and maintenance, cost for equipment replacement, etc. [3,16]. Annual energy exchange cost, which is the difference between the cost of microgrids purchasing electricity from the large grid and the revenue from microgrids selling electricity every year, is also added in the cost of N PV of the life cycle [7]. In this work, the economic object is composed of the four parts mentioned above and can be described in Eqs (6)-(9), as represented below.
c(r, l) = r(1 + r) l [(1 + r) l − 1] (8) Where, C wt , C pv , C bat are respectively the equipment investment cost of the wind turbine, photovoltaic and battery; C OM wt , C OM pv , C OM bat are the operation and maintenance costs of the wind turbine, photovoltaic and battery respectively; C R wt , C R pv , C R bat are the replacement cost of the wind turbine, photovoltaic and battery respectively; c(r,l) is the present value coefficient, which is related to the discount rate r and the service life l of microgrid; N DG is the number of power types; x i is the number of i-th power supply; and P i is the installed capacity of i-th power supply.

Load power shortage rate of microgrid
The ability of microgrid to meet the load demand in microgrid independently is defined as the self-balancing rate of microgrid, that is, the proportion of annual power supply of microgrid equipment to the annual power consumption of load, which is shown as the equation below [3,7].
Where P S (t) is the power generated from the microgrid in t time to meet the load demand P L (t). The larger the self-power supply capacity is, the better, thus, the target function can be represented by the rate of load power shortage, which can be described in the form of the following equation:

Constrains
(1) Number of micro-sources constrain 0 ≤ N wt ≤ N wt_max (12) 0 ≤ N pv ≤ N pv_max (13) (2) Limitation of interaction ability with the external power grid To prevent the influence on the stability of the external power grid, the power interaction limit between the microgrid and external power grid is set.
(3) Power balance constrain be described below.
Where P wt (t) is the power generated by wind turbine; P pv (t) is the power generated by the Photovoltaic; P bat (t) is the power generated by the battery; P exc (t)is the power exchanged with external power grid; P exe (t) is the overflow power; and P L (t) is the load demand in time t.
(4) Battery charging and discharging power and depth constraint

Optimisation method
Multi-agent search strategy has attracted much concern for its promising computational model in optimisation problems in these years. The agent can be seen as a physical or abstract entity, which has the perception, interaction and problem-solving ability [17]. Multiple agents compose the MAS. The MAS has remarkable features, such as autonomy, distribution, coordination, etc. By virtue of self-organisation ability, learning ability and reasoning ability, the multi-agent search strategy for optimisation problems achieved good results.
In this section, a DMAMOEA for a multi-objective microgrid allocation optimal problem is proposed based on the concept of the Pareto method. In this algorithm, several operators for a multi-objective problem are designed, such as neighbourhood Pareto preferred operator, neighbourhood differential evolution operator, mutation operator, etc. By these operators, the agents in MAS interact with each other and produce feasible solutions for the multi-objective microgrid allocation problem. The non-dominated solutions produced in each generation are kept in the archive set. To guarantee the uniformity of the archive set, the solutions with larger crowding distances are further optimised. The simulation results demonstrated the effectiveness of DMAMOEA.

The structure of MAS for multi-objective optimisation
In the structure of MAS, each agent stands for a feasible solution, which is a real-valued vector. All the agents are fixed on a squared network. The structure of the network is shown in Fig. 1. Each agent can only interact with the agent's neighbourhood. Fig. 1 The structure of multi-agent system The neighbourhood of Agent L i j can be depicted as follows: where

Neighbourhood Pareto preferred operator
The neighbourhood Pareto preferred operator is designed to determine the dominance relations among the individuals in the local environment of agent L i j , including itself: the agent L i j and the agent in its local environment are compared with each other to find the Pareto solution. If the agent L i j dominates the agents in its local environment or there is no dominance relation, the agent L i j is the best solution in this local environment.

Neighbourhood differential evolution operator
Randomly select three solutions in the neighbourhood agent L i j and perform the mutation operation with them. If the selected solutions in neighbourhood are the upper, left and right positions of L i j , the mutation performance can be described in the equation below.
Where, in this equation, L k (i−1) j is the k-th variable of the upper agent of L i j ; L i( j−1) k is the k-th variable of the left agent of L i j ; L i( j+1) k is the k-th variable of the right agent of L i j , and is the k-th variable in newly generated individual L i j,mutation .
In the mutation process, if L i j,mutation exceeds the upper and lower limit, the value of the variable should be replaced by 0 or 1.
After the mutation operation, the crossover operation is executed. This operation can be described in the equation below. In this equation, L k i j is the k-th variable in L i j , L k i j,mutation is the k-th variable in L i j,mutation , and L k i j,de is the k-th variable in newly generated individual L i j,de . In this operation, at least one variable in L i j,de should be taken from L k i j,mutation .
After the crossover mutation, the dominance relationship between L i j and L i j,de is compared. The individuals with higher dominance levels should be selected to replace L i j .

Mutation operator
To further maintain the distribution of the population, the mutation operation is performed on all the variables of agents in the lattice with mutation probability P m , by which a small number of new agents will take place. This process can be described in the equation below.
Where, q i is the original variable; e i is the variable after mutation operation; r is a random number in the range (0,1); is the perturbation amplitude;is the perturbation variable, the value of which is the sum of all the values selected in set X = (2 −0 ,2 −1 ,. . . ,2 −m ) with probability 1/m.

Distributed maintenance strategy
To maintain the distribution of the solution in the archive set, crowding distance is introduced to estimate the crowding degree of the solutions, as shown in Fig. 2. Taking the i-th point in the archive set as an example, the crowding distance is the average side length of a cuboid, which is composed of points near the i-th. The variables of solutions with the larger crowding distance in the set plus Gaussian perturbation and new solutions are produced. This process can be described in the equation below. In this equation, l i is the variable in the solutions, and e i ' is the variable in the new solutions. The new solutions are then compared to the solution in the set. The dominated solutions are eliminated and the dominating ones are kept. By this operation, the distribution of the solutions is more uniform.

Procedure of DMAMOEA
The procedure of DMAMOEA can be summarised as follows: Step1: Let t=0 and generate the population randomly, Q (t)= {X 1 t , X 2 t ,. . . , X n t }, where X i t represents the ith individual and n is the number of individuals; Step 2: Distribute all the individuals on a √ n × √ n lattice; Step 3: If the iteration number reaches the set value (t = maxgen), terminate the procedure and output the archive set P best ; Step 4: Implement the neighbourhood Pareto preferred operator on the agents on the lattice sequentially, and the best solution maxL i j in the local environment of L i j is compared with solutions in the archive set. In this process, the solutions dominated by maxL i j will be eliminated from P best , and maxL i j will be added to P best if it is not dominated by any solution.
Step 5: Implement the neighbourhood differential evolution operator on each agent; Step 6: Implement the mutation operator on Q (t) with probability P m , and generate a new population Q(t+1); Step7: Calculate the crowding distance of all the solutions in P best , further optimise the solutions with a larger crowding distance. If the number of solutions in P best exceeds the size (num_of_solutions > size_of _set), then the solutions will be sorted by the crowding distance, and the solutions with the largest crowding distance are kept in P best . If the iteration number is less than the set value, go back to Step 3. This procedure is shown in Fig. 3. The capacity of the fan is 30 kW, the capacity of the photovoltaic equipment is 1 kW, and the single battery capacity is 50 kW h. The value of initially installing cost, operating and maintaining cost, the replacement cost of each item of equipment is shown in Table 1. The upper and lower limit of battery power is 0.3 and 0.8, and the default initial power is 0.5. The interactive power consumption with the grid is 10% of the maximum power consumption in the period. Compared with other multi-objective optimisation algorithms, the NSGA-II algorithm has some advantages in solution efficiency and result division [18]. Therefore, the classical NSGA-II algorithm and the algorithm proposed in this article are used to optimise the problem.
In this study, the running CPU is Core i5-6300 CPU, the memory is 8 Gb, and MATLAB software is also used. In terms of parameter setting, the DMAMOEA is set to 16 agents, the archive set scale is 200 and the number of iterations is 100; the population number of NSGA-II is 200, and the number of iterations is 70. The simulation result is shown in Fig. 8.
As can be seen from Fig. 8, the results of the two algorithms coincide. To evaluate the approximation degree of the results of the two algorithms to the real Pareto optimal solution, generation distance is used to evaluate the results of the two algorithms. The equation of generation distance is described as follows [18]: Where, P is the solutions obtained by the proposed algorithm and P* is the ideal solution set. In this study, the ideal solution set is represented by the non-dominated solution set of the two solution sets obtained by the two algorithms. Generation distance is the average value of the sum of the minimum distance of each solution in the solution set and the solution in the ideal solution set. The smaller the value is, the better the result is. Fig. 9 is a column comparison diagram of generation distance obtained by DMAMOEA and NSGA-II running 10 times respectively. In terms of the operation time, DMAMOEA takes an average of 52 s, while NSGA-II takes about 66 s. Therefore, the DMAMOEA is superior to the traditional NSGA-II in terms of operation time and the approximation of the final solution set.      Table 2 shows some representative results of DMAMOEA algorithm. Taking Solution 1 as an example, using the Environmental Benefit Analysis Method [7], the representative results are obtained and the results are shown in Table 3. From Tables 1 and 2, the following conclusions can be drawn: (1) In terms of Solution 1 and Solution 2, there is some difference in the number of micro sources. Between the two solutions, the number of wind turbines and batteries in Solution 1 is more than that in Solution 2, while the photovoltaic equipment is less than that in Solution 2. Therefore, although the investment increases, the dependence of microgrids on the external network is reduced.
(2) From Solution 3 and Solution 4, the number of wind turbines, photovoltaic equipment and batteries in Solution 3 is increased, especially the number of batteries is increased from 63 to 231. Therefore, the energy Fig. 9 Comparison of Gd Distance DEMAMOME and NSGA-II  storage capacity of the microgrid is further enhanced, so that the microgrid can meet its load demand much better.
(3) Considering the environmental benefits, and taking Solution 1 as an example, the environmental benefits obtained are shown in Table 3. In this table, through the implementation of microgrids, the total emission of pollutants is reduced by 89,600 tons, the environmental protection cost is saved by US $1,826,100, and the environmental protection benefit is obvious, which fully reflects the benefit of the implementation.

Conclusions
In this article, a DMAMOEA was proposed to optimise the two objectives allocation of microgrid system with photovoltaic, wind power and battery. The two optimisation objectives are the economy index which including investment, maintenance and replacement of the micro sources and the rate of load power shortage index which reflects the degree of dependence on the external power grid. At the end of the article, the optimisation results of this proposed algorithm are compared with the classic algorithm NSGA-II, and the following conclusions are reached: (1) A DMAMOEA is designed by combining differential evolution strategy and multi-agent technology. (2) In the capacity optimisation model of microgrid, two goals of economy and reliability are considered.
(3)The optimisation results of the designed algorithm are compared with that of the classic multi-objective evolutionary algorithm NSGA-II, which shows that DMAMOEA has a shorter solution time and a better solution set than the NSGA-II algorithm.
T h i s p a g e i s i n t e n t i o n a l l y l e f t b l a n k