Optimization methods and landscapeThe analysis of structures (landscapes) of MOPs and the performance assessment
of resolution methods are significant topics in the design of optimization
methods. The effectiveness of metaheuristics depends on the properties of the
landscape (roughness, convexity, etc). The notion of landscape has been first
described in
The landscape is defined by a neighborhood operator and can be represented by a
graph G = (V, E). The vertices represent the solutions of the problem and an
edge (e1, e2) exists if the solution e2 can be obtained by an
application of the neighborhood operator on the solution e1. Then,
considering this graph as the ground floor, we elevate each solution to an
altitude equals to its cost. We obtain a surface, or landscape, made of peaks,
valleys, plateaus, cliffs, ...The problem lies in the difficulty to have a
realistic view of this landscape.
Analysis of the structure of a problemAs others, we believe that the main point of interest in the domain of
combinatorial optimization is not the design of the best algorithm for a large
number of problems but that it is to search for the most adapted method to an
instance or a set of instances of a given problem. Therefore, we guess that no
ideal metaheuristic, designed as a black-box, may exist.
Indeed, the first studies realized in our research group on the analysis of
landscapes of different mono-objective combinatorial optimization problems
(traveling salesman problem, quadratic assignment problem) have showed that
not only different problems correspond to different structures but also that
different instances of the same problem correspond to different structures.
For instance, we have realized a statistical study of the landscapes of the
quadratic assignment problem. Some indicators, which characterize the
landscape of an instance, have been proposed and a taxonomy of the instances
including three classes has been deduced. Hence it is not enough to adapt the
method to the problem under study but it is necessary to specialize it
according to the type of instance treated.
So in its studies of mono-objective problems, the OPAC research group has
introduced into the resolution methods some information about the problem to
be solved. The landscapes of some combinatorial problems have been studied in
order to investigate the intrinsic natures of their instances. The resulting
information have been inserted into an optimization strategy and allowed the
design of efficient and robust hybrid methods. The extension of these studies
to multi-objective problems is a part of the DOLPHIN project
Performance assessmentThe DOLPHIN project is also interested in the performance assessment of
multi-objective optimization methods. Nowadays, statistical techniques
developed for mono-objective problems are used with adaptation to the
multi-objective case. Nevertheless, specific tools are necessary in many
cases: for example, the comparison of two different algorithms is relatively
easy in the mono-objective case - we compare the quality of the best solution
obtained in a fixed time, or the time needed to obtain a solution of a certain
quality. The same idea can not be immediately transposed to the case where the
output of the algorithms is a set of solutions having several quality
measures, and not a single solution.
Various indicators have been proposed in the literature for evaluating the
performance of multi-objective optimization methods but no indicator seems to
outperform the others
These metrics and others (generational distance, spacing, ...) are integrated
in the open software GUIMOO developed within the framework of the DOLPHIN
project. This software is dedicated to the visualization of landscapes (2D and
3D) for multi-objective optimization and the performance analysis by the use
of special metrics.
GoalsOne of the main issue in the DOLPHIN project is the landscape study of the
multi-objective problems and the performance assessment of multi-objective
optimization methods to design efficient and robust resolution methods:
Landscape study:
The goal here is to extend the study of landscapes of the mono-objective
combinatorial optimization problems to multi-objective problems in order to
determine the structure of the Pareto frontier and to integrate this knowledge
about the
problem structure in the design of resolution methods.
This study has been initiated for the bi-objective flow-shop problem. We have
studied the convexity of the frontiers obtained in order to show the interest
of our Pareto approach compared to an aggregation approach, which only allows
to obtain the Pareto solutions situated on the convex hull of the Pareto front
(supported solutions).
Our preliminary study of the landscape of the bi-objective flow-shop problem
shows that the supported solutions are very closed to each other. This remark
leads us to improve an exact method initially proposed for bi-objective
problems. Furthermore, a new exact method able to deal with any number of
objectives has been designed.
Performance assessment:
The goal here is to extend GUIMOO in order to provide efficient visual and
metric tools for evaluating the assessment of multi-objective resolution
methods.
Hybrid multi-objective optimization methodsThe success of metaheuristics is based on their ability to find efficient
solutions in a reasonable time
The DOLPHIN project deals with ``a posteriori'' multi-objective
optimization where the set of Pareto solutions (solutions of best compromise)
have to be generated in order to give to the decision maker the opportunity to
choose the solution that interests him.
Population based methods, such as evolutionary algorithms, are well fitted for
multi-objective problems as they work with a set of solutions
[http://www.lania.mx/~ccoello/EMOO/EMOObib.html]) that
contains more than 1600 references. One of the objectives of the project is to
propose advanced search mechanisms for intensification and diversification.
These mechanisme have been designed in an adaptive manner, since their
efficiency is related to the landscape of the MOP.
In order to assess the performances of the proposed mechanisms, we always
proceed in two steps: first, experiments are carried out on academic problems,
for which some best known results exist; second, we use real industrial
problems to cope with large and complex MOPs. The lack of references in terms
of optimal or best know Pareto set is a major problem. Therefore, the obtained
results in this project and the test data sets will be available at the URL
[http://www.lifl.fr/OPAC] at Benchmarks.
Cooperation of metaheuristicsIn order to cope with advantages of the different metaheuristics, an
interesting idea is to combine them. Indeed, the hybridization of
metaheuristics allows the cooperation of methods having complementary
behaviors. The efficiency and the robustness of such methods depend on the
balance between the exploration of the whole search space and the exploitation
of interesting areas.
Hybrid metaheuristics have received considerable interest these recent years in
the field of combinatorial optimization. A wide variety of hybrid approaches
have been proposed in the literature and give very good results on numerous
single objective optimization problems, that are either academic (traveling
salesman problem, quadratic assignment problem, scheduling problem, ...) or
real-world problems. This efficiency is due to combinations of single solution
based methods (iterative local search, simulated annealing, tabu search...)
with population based methods (genetic algorithms, ants search, scatter
search...). A taxonomy of hybridization mechanisms may be found in
LRH class - Low-level Relay Hybrid: This class groups
algorithms in which a given metaheuristic is embedded into a single-solution
metaheuristic. Few examples from the literature belong to this class.
LTH class - Low-level Teamwork Hybrid: In this class, a
metaheuristic is embedded into a population-based metaheuristic in order to
exploit strengths of single-solution and population-based metaheuristics.
HRH class - High-level Relay Hybrid: Here, self contained
metaheuristics are executed in a sequence. For instance, a population-based
metaheuristic is executed to locate interesting regions and then a local
search is performed to exploit these regions.
HTH class - High-level Teamwork Hybrid: This scheme
involves several self-contained algorithms performing a search in parallel and
cooperating. An example will be the island model, based on GAs, where the
population is partitioned into small subpopulations and a GA is executed per
subpopulation. Some individuals can migrate between subpopulations.
Let us notice, that if hybrid methods have been studied in the mono-criterion
case, their application in the multi-objective context is not yet widely
spread. The objective of the DOLPHIN project is to integrate specificities of
multi-objective optimization into the definition of hybrid models
Cooperation between metaheuristics and exact methodsUntil now only few exact methods have been proposed to solve multi-objective
problems. They are based either on a Branch-and-bound approach, on the
algorithm A* or on dynamic programming. However, those methods are limited
to two objectives and are, most of the time, not able to be used on the
complete problem, which is often large scale MOP. Therefore, sub search spaces
have to be defined in order to be able to use exact methods. Hence, in the
same manner than hybridization of metaheuristics, the cooperation of
metaheuristics and exact methods is also a main issue in this this project.
Indeed, it allows to use the exploration capacity of metaheuristics as well as
the intensification ability of exact methods that are able to find optimal
solution(s) in a restricted search space. Sub search spaces will have to be
defined along the search. Such strategies can be found in the literature, but
they are only applied to mono-objective academic problems.
We have extended the previous taxonomy for hybrid metaheuristics to the
cooperation between exact methods and metaheuristics. Using this taxonomy, we
are investigating cooperative multi-objective methods. In this context,
several types of cooperations may be considered, according to the way the
metaheuristic and the exact method cooperates. For instance, a metaheuristic
can use an exact method for intensification or an exact method can use a
metaheuristic to reduce the search space).
Moreover, a part of the DOLPHIN project deals with studying exact methods in
the multi-objective context in order: i) to be able to solve small size
problems and to validate proposed heuristic approaches; ii) to have more
efficient/dedicated exact methods that can be hybridized with metaheuristics.
In this context, the use of parallelism will push back limits of exact methods
which will be able to explore larger size search space
GoalsBased on the previous works on multi-objective optimization, it appears that to
improve metaheuristics, it becomes essential to integrate knowledge about the
problem structure. This knowledge can be gained during the search. This would
allow to adapt operators that may be specific for multi-objective optimization
or not. The goal here is to design auto-adaptive methods that are able to
react to the problem structure. Moreover, regarding the hybridization and the
cooperation aspects, the objectives of the DOLPHIN project are to deepen those
studies as follows:
improve metaheuristics, it becomes essential to integrate knowledge about the
problem structure, that we may get during the execution. This would allow to
adapt operators that may be specific for multi-objective optimization or not.
The goal here is to design auto-adaptive methods that are able to react to the
problem structure.
Design of cooperative metaheuristics: Previous studies show the
interest of hybridization for a global optimization and the importance of
problem structure study for the design of efficient methods. It is now
necessary to generalize metaheuristic hybridization and to propose adaptive
hybrid models that may evolve during the search while selecting the
appropriate metaheuristic. Multi-objective aspects have to be introduced in
order to cope with specificities of multi-objective optimization.
Design of cooperative schemes between exact methods and
metaheuristics: Once the study on possible cooperation schemes will be ended,
we will have to test and compare them in the multi-objective context.
works on parallel metaheuristics allow us to speed up the resolution of large
scale problems. It could be also interesting to study the robustness of the
different parallel models (in particular in the multi-objective case) and to
propose rules that determine, given a specific problem, which kind of
parallelism to use.
Of course these goals are not disjoined and it will be interesting to
simultaneously use hybrid metaheuristics and exact methods. Moreover, those
advanced mechanisms may require the use of parallel and distributed computing
in order to easily make evolve simultaneously cooperating methods and to speed
up the resolution of large scale problems.
proceed in two phases: validation on academic problems, for which some best
known results exist and use on real problems (industrial) to cope with problem
size constraints.
distributed multi-objective aspects in the ParadisEO Platform (see the
paragraph on software platform).
Parallel multi-objective optimization: models and software frameworksParallel and distributed computing may be considered as a tool to speedup the
search to solve large MOPs and to improve the robustness of a given method.
Moreover, the joint use of parallelism and cooperation allows the improvement
of the quality of the obtained Pareto sets. Following this objective, we will
design and implement parallel models for metaheuristics (evolutionary
algorithms, tabu search approach) and for exact methods (branch and bound
algorithm, branch and cut algorithm) for solving different large MOPs.
One of the goal of the DOLPHIN project is to integrate the developed parallel
models into software frameworks. Several frameworks for parallel distributed
metaheuristics have been proposed in the literature. Most of
them
focus only either on evolutionary algorithms or on local search methods. Only
few frameworks are dedicated to the design of both of the two families of
methods. On the other hand, existing optimization frameworks either do not
provide parallelism at all or just supply at most one parallel model. In this
project, a new framework for parallel hybrid metaheuristics is proposed, named
Parallel and Distributed Evolving Objects (ParadisEO) based on EO. The
framework provides in a transparent way the hybridization mechanisms presented
in the previous section, and the parallel models described in the next
section. Concerning the developed parallel exact methods for MOPs, we will
integrate them into well-known frameworks such as COIN.
Parallel modelsAccording to the family of addressed metaheuristics, we may distinguish two
categories of parallel models: parallel models managing a single solution and
parallel models that handle a population of solutions. The major single
solution-based parallel models are the following: the parallel
neighborhood exploration model and the multi-start model.
The parallel neighborhood exploration model is basically a ``low
level" model that splits the neighborhood into partitions that are explored
and evaluated in parallel. This model is particularly interesting when the
evaluation of each solution is costly and/or when the size of the neighborhood
is large. It has been successfully applied to the mobile network design
problem (see Application section).
The multi-start model consists in executing in parallel several
local searches (that may be heterogeneous), without any information exchange.
This model raises particularly the following question: is it equivalent to
execute k local searches during a time t than executing a single local
search during k×t? To answer this question we tested a multi-start
Tabu search on the quadratic assignment problem. The experiments have shown
that the answer is often landscape-dependent. For example, the multi-start
model may be well-suited for landscapes with multiple
basins.
On the other hand, parallel models that handle a population of solutions are
mainly: the island model, the central model and the
distributed evaluation of a single solution. Let us notice that the last
model may also be used with single-solution metaheuristics.
In the island model, the population is split into several
sub-populations distributed among different processors. Each processor is
responsible of the evolution of one sub-population. It executes all the steps
of the metaheuristic from the selection to the replacement. After a given
number of generations (synchronous communication), or when a convergence
threshold is reached (asynchronous communication), the migration process is
activated. Then, exchanges of solutions between sub-populations are realized,
and received solutions are integrated into the local sub-population.
The central (Master/Worker) model allows to keep the sequentiality
of the original algorithm. The master centralizes the population and manages
the selection and the replacement steps. It sends sub-populations to the
workers that execute the recombination and evaluation steps. The latter return
back newly evaluated solutions to the master. This approach is efficient when
the generation and evaluation of new solutions is costly.
The distributed evaluation model consists in a parallel evaluation
of each solution. This model has to be used when, for example, the evaluation
of a solution requires access to very large databases (data mining
applications) that may be distributed over several processors. It may also be
useful in a multi-objective context, where several objectives have to be
computed simultaneously for a single solution.
As these models have now been identified, our objective is to study them in the
multi-objective context in order to use them advisedly. Moreover, these models
may be merged to combine different levels of parallelism and to obtain more
efficient methods
The ParadisEO frameworkIn the last years, loosely and closely coupled clusters of workstations have
become a real architectural alternative to traditional supercomputing
environments. As a matter of fact, workstations are more and more powerful
while they are less and less expensive. In addition, interconnection networks
have known a great technological evolution that makes them more speedy
(Myrinet, GigaEthernet, etc.). Our research group has developed a parallel
adaptive programming environment named MARS (Multi-user Adaptive Resource
Scheduler). MARS provides for the programmers an APIApplication
Programming Interface. to simplify the development of parallel applications,
and a support for their scheduling on adaptive networks and clusters of
heterogeneous workstations. The MARS open source has been made available on
the Web at the disposal of the users for free download and use. With such an
experience in the development of open source software, we are developping the
ParadisEO software. This framework is detailed further in this document.
GoalsOur objectives focus on these issues are the following:
Design of parallel models for metaheuristics and exact methods for
MOPs: We will develop parallel cooperative metaheuristics (evolutionary
algorithms and local search such as Tabu search) for solving different large
MOPs. Moreover, we are designing a new exact method, named PPM (Parallel
Partition Method), based on branch and bound and branch and cut algorithms.
Finally, some parallel cooperation schemes between metaheuristics and exact
algorithms have to be used to solve MOPs in an efficient manner.
Integration of the parallel models into software frameworks: The
parallel models for metaheuristics will be integrated in the ParadisEO
software framework. The proposed multi-objective exact methods must be first
integrated into standard frameworks for exact methods such as COIN and BOB++.
A coupling with ParadisEO is then needed to provide hybridization
between metaheuristics and exact methods.
Efficient deployment of the parallel models on different parallel
and distributed architecture including GRIDs: The designed algorithms and
frameworks will be efficiently deployed on non-dedicated networks of
workstations, dedicated cluster of workstations and SMP (Symetric
Multi-processors) machines. For GRID computing platforms, peer to peer (or
P2P) middlewares (XtremWeb-Condor) will be used to implement our frameworks.
For this purpose, the different optimization algorithms may be re-visited for
their efficient deployment (see
-constraint approach, in order to divide the search space into well-balanced partitions. The third stage consists in finding the other efficient solutions in each partition. This method has a parallel behavior which may increase its performances. It has been applied on a bi-objective flow-shop and gave interesting results.