What is multi criteria decision making?
When more than one criterion speaks at once, a choice becomes a justification. This guide covers the chain from the decision matrix to method selection.
What is multi criteria decision making?
Multi criteria decision making is a family of methods that rank or select among options against more than one criterion. Methods do not remove the conflict, they turn the preference into something numerical and auditable.
MCDA (multi-criteria decision analysis) is the other abbreviation for the same task (Hwang and Yoon, 1981).
A decision problem can be framed in three ways:
- Choice: identifying one option, or a small subset, among the alternatives.
- Ranking: ordering all options from best to worst.
- Classification: placing options into predefined categories, for example accept, review, reject.
Method choice follows from this framing: the ELECTRE family compares through an outranking relation (Roy, 1968), others compare options through distance to a reference point.
What is the difference between a single criterion choice and a multi criteria choice?
In a single criterion choice the result is not open to debate. In a multi criteria choice criteria cannot be converted into each other; depending on the trade off accepted, different options can be justified.
Two concepts make this concrete: dominance and trade off preference. An option dominates another if it is better or equal on every criterion and strictly better on one; the decision maker's trade off preference orders the rest.
Criteria have different units and directions; values are brought onto a common scale and cost oriented criteria are reversed. This step is normalisation.
What does a decision matrix consist of?
A decision matrix carries options in its rows and criteria in its columns. Each cell is accompanied by the criterion's direction, unit and weight.
| Option | Price (TRY, cost) | Lead time (days, cost) | Defect rate (%, cost) | Capacity (units, benefit) |
|---|---|---|---|---|
| Supplier A | 118 | 12 | 1.4 | 9,000 |
| Supplier B | 104 | 21 | 2.6 | 14,000 |
| Supplier C | 131 | 9 | 0.8 | 7,500 |
Example matrix; the numbers are constructed, not measured.
In the table no option dominates the other: B is the cheapest with the highest capacity, C is the fastest with the lowest defect rate.
Three things are written down when the matrix is built: the measurement definition, the source of each value, how missing cells are filled.
A criterion stays if it makes a real difference and can be measured. Two criteria standing in for each other weight the same information twice.
How are criteria weights determined?
A weight is a criterion's relative importance, and it comes from three routes: a direct statement, pairwise comparison, objective measures. The source must be written down, since weights usually change a ranking the most.
Expert opinion and direct scoring
The simplest route is asking the decision maker to weight criteria directly: distributing a hundred points across them, or scoring each and normalising to one. Its weak point is that consistency cannot be checked.
AHP and pairwise comparison
The analytic hierarchy process splits the problem into goal, criteria and option layers and asks about importance through pairwise comparison. Weights are derived from that matrix (Saaty, 1980).
Its distinguishing contribution is the consistency check: contradictory ratings are caught by the consistency ratio, with 0.10 as the threshold. The cost is that comparisons grow quickly with the criteria.
Objective methods and entropy
Objective weighting computes the weight from the decision matrix itself. In the entropy approach, values close together mean a criterion is not distinguishing options, so it gets a low weight.
An objective weight says a criterion is distinguishing in the data, not that it matters. A criterion such as safety may get a low weight if options are similar; objective and subjective weights are therefore often used together.
Which method is used in which situation?
Method choice depends on the shape of the problem: distance, compromise and outranking methods answer different needs.
| Method | Core idea | When it fits | Source |
|---|---|---|---|
| AHP | Splits the problem into layers, measures importance through pairwise comparisons. | When criteria are hierarchical and importance comes from an expert. | (Saaty, 1980) |
| TOPSIS | Measures distance to the ideal and negative ideal solution together. | When data is numerical and complete, trade offs freely accepted. | (Hwang and Yoon, 1981) |
| VIKOR | Looks for a compromise that balances group utility with individual regret. | When criteria conflict sharply and trade offs need to be bounded. | (Opricovic and Tzeng, 2004) |
| PROMETHEE | Defines a preference function per criterion, turns comparisons into an outranking flow. | When a threshold can be defined per criterion. | (Brans and Vincke, 1985) |
| ELECTRE | Uses concordance and discordance tests to check outranking. | When weak options need to be eliminated and incomparability preserved. | (Roy, 1968) |
The methods in the table are tools for different jobs: AHP produces weights, the others take weights and compare options.
The real distinction between methods is compensation: in TOPSIS, poor performance can be offset, in outranking methods this is limited.
The result does not have to be a single winner; outranking methods can leave two options unordered.
What is done if there is uncertainty?
If there is uncertainty, values are written as a range or a membership degree instead of a single number. Fuzzy sets allow an element to partly belong to a set (Zadeh, 1965).
The practical counterpart of the fuzzy approach is the linguistic scale: an expert picks a term such as very good, good, or medium rather than a score. The result is then defuzzified into a comparable value.
Interval and grey numbers carry a different uncertainty: the value is unknown, but its bounds are. Saying a quote is between 100 and 130 is more honest than averaging it to 115.
Modelling uncertainty does not reduce it, it makes it visible.
Why does the same data give a different ranking under different methods?
Because every method carries a different preference assumption: normalisation form and compensation degree vary. The same matrix can produce different rankings; the difference is not an error.
Four main sources pull rankings apart:
- Normalisation: there is more than one way to bring values onto a common scale, each distorts distance differently.
- Compensation: methods that allow compensation favour the higher average, methods that do not favour the option with no weak side.
- Threshold definition: methods with thresholds deliberately ignore small differences; methods without them can let those differences decide.
- Rank reversal: when an option is added or removed, the order of the rest can change.
The correct response is to choose the method before the ranking and report all results if more than one was run.
waiting on two runs
Filling the decision matrix: row A1 in progress.
flow 3/8How is an MCDM analysis defended?
The defence rests on four things: method rationale, the weights' source, a sensitivity analysis, reproducibility. Unwritten, what exists is a preference, not a finding.
Sensitivity analysis is the most often skipped, and the easiest to do: raise and lower each weight and find where the top option changes.
Peng (2011) defines the minimum as publishing data and code with the result; here that means the matrix, weight vector and parameters in the appendix. FAIR (Wilkinson et al., 2016) and PROV (W3C, 2013) frame that appendix.
A defensible report contains the following:
- Problem framing: whether it is choice, ranking or classification, and who the decision maker is.
- Criteria list: each criterion's definition, direction, unit and why it is on the list.
- Decision matrix: raw values, their source and how missing data was handled.
- Weights: the method, the source, and the consistency check's result if there was one.
- Method justification: why the chosen preference assumption fits the problem.
- Sensitivity analysis: how the ranking behaves as weights and data change.
- Reproduction appendix: the matrix, weight vector, parameters and software version.
Which tools do this work?
This chain can be run by hand. What gets hard is tracking which step ran with which input.
Which other guides cover this topic?
References
- Brans, J. P. and Vincke, P. (1985). A preference ranking organisation method. Management Science, 31(6), 647-656.
- Hwang, C. L. and Yoon, K. (1981). Multiple Attribute Decision Making: Methods and Applications. Springer.
- Opricovic, S. and Tzeng, G. H. (2004). Compromise solution by MCDM methods. European Journal of Operational Research, 156(2), 445-455.
- Peng, R. D. (2011). Reproducible research in computational science. Science, 334(6060), 1226-1227.
- Roy, B. (1968). Classement et choix en presence de points de vue multiples. RIRO, 2(8), 57-75.
- Saaty, T. L. (1980). The Analytic Hierarchy Process. McGraw-Hill.
- W3C (2013). PROV-DM: The PROV Data Model. W3C Recommendation.
- Wilkinson, M. D. et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018.
- Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338-353.
Let's build your decision analysis together.
If your institution has a multi criteria decision problem, write in for a demo or an institutional licence.