< previous | table of contents

  1. New Zealand Public Health and Disability Act, Stat. 2000 No 91 (2000).
  2. Drummond M, Sculpher M, Torrance G, O'Brien B, Stoddart G. Methods for the Economic Evaluation of Health Care Programmes. 3rd ed. ed. Oxford: Oxford University Press; 2005.
  3. Pharmaceutical management Agency (PHARMAC). Pharmacology and Therapeutics Advisory Committee (PTAC) 2015. Available from:
  4. TreeAge Pro. Williamstown, MA: TreeAge Software.
  5. Pharmaceutical management Agency (PHARMAC). PHARMAC's performance 2015. Available from:
  6. Grocott R. Applying Programme Budgeting Marginal Analysis in the health sector: 12 years of experience. Expert review of pharmacoeconomics & outcomes research. 2009 Apr;9(2):181-7.
  7. Weinstein MC, Toy EL, Sandberg EA, Neumann PJ, Evans JS, Kuntz KM, et al. Modeling for health care and other policy decisions: uses, roles, and validity. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2001 Sep-Oct;4(5):348-61.
  8. Soto J. Health economic evaluations using decision analytic modeling. Principles and practices--utilization of a checklist to their development and appraisal. Int J Technol Assess Health Care. 2002 Winter;18(1):94-111.
  9. Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health technology assessment. 2004 Sep;8(36):iii-iv, ix-xi, 1-158.
  10. Sculpher M, Fenwick E, Claxton K. Assessing quality in decision analytic cost-effectiveness models. A suggested framework and example of application. PharmacoEconomics. 2000 May;17(5):461-77.
  11. Briggs A, Sculpher M. An introduction to Markov modelling for economic evaluation. PharmacoEconomics. 1998 Apr;13(4):397-409.
  12. Coyle D. Statistical analysis in pharmacoeconomic studies. A review of current issues and standards. PharmacoEconomics. 1996 Jun;9(6):506-16.
  13. Revicki DA, Frank L. Pharmacoeconomic evaluation in the real world. Effectiveness versus efficacy studies. PharmacoEconomics. 1999 May;15(5):423-34.
  14. Siegel JE, Torrance GW, Russell LB, Luce BR, Weinstein MC, Gold MR. Guidelines for pharmacoeconomic studies. Recommendations from the panel on cost effectiveness in health and medicine. Panel on cost Effectiveness in Health and Medicine. PharmacoEconomics. 1997 Feb;11(2):159-68.
  15. Saint S, Veenstra D, Sullivan S. The use of meta-analysis in cost-effectiveness analysis. Issues and recommendations. PharmacoEconomics. 1999 January;15(1):1-8.
  16. Jackson R, Ameratunga S, Broad J, Connor J, Lethaby A, Robb G, et al. The GATE frame: critical appraisal with pictures. Evidence-based medicine. 2006 Apr;11(2):35-8.
  17. Gray R, Bentham P, Hills R, Sellwood E, Stowe R, on behalf of the AD2000 Trial Steering Committee. Improvements in functional ability with galantamine in Alzheimer’s have not yet been established. BMJ. 2001.
  18. Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995 Aug 19;311(7003):485.
  19. Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002 Jan 19;359(9302):248-52.
  20. Freemantle N, Calvert M, Wood J, Eastaugh J, Griffin C. Composite outcomes in randomized trials: greater precision but with greater uncertainty? Jama. 2003 May 21;289(19):2554-9.
  21. Lauer MS, Topol EJ. Clinical trials--multiple treatments, multiple end points, and multiple lessons. Jama. 2003 May 21;289(19):2575-7.
  22. Pharmaceutical Benefits Advisory Committee. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee (Version 4.4). 2013.
  23. Weinstein MC, O'Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices--Modeling Studies. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2003 Jan-Feb;6(1):9-17.
  24. Statistics New Zealand. Period Life Tables 2015. Available from:
  25. Brennan A, Akehurst R. Modelling in health economic evaluation. What is its place? What is its value? PharmacoEconomics. 2000 May;17(5):445-59.
  26. Taylor RS, Iglesias CP. Assessing the clinical and cost-effectiveness of medical devices and drugs: are they that different? Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2009 Jun;12(4):404-6.
  27. Sorenson C, Tarricone R, Siebert M, Drummond M. Applying health economics for policy decision making: do devices differ from drugs? Europace : European pacing, arrhythmias, and cardiac electrophysiology : journal of the working groups on cardiac pacing, arrhythmias, and cardiac cellular electrophysiology of the European Society of Cardiology. 2011 May;13 Suppl 2:ii54-8.
  28. Drummond M, Griffin A, Tarricone R. Economic evaluation for devices and drugs--same or different? Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2009 Jun;12(4):402-4.
  29. Buxton MJ, Drummond MF, Van Hout BA, Prince RL, Sheldon TA, Szucs T, et al. Modelling in economic evaluation: an unavoidable fact of life. Health economics. 1997 May-Jun;6(3):217-27.
  30. Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D'Amico R, et al. Indirect comparisons of competing interventions. Health technology assessment. 2005 Jul;9(26):1-134, iii-iv.
  31. Wells G, Sultan S, Chen L, Khan M, Coyle D. Indirect evidence: indirect treatment comparisons in meta-analysis. In: Health CAfDaTi, editor. Ottawa2009.
  32. Cook DI, Gebski VJ, Keech AC. Subgroup analysis in clinical trials. The Medical journal of Australia. 2004 Mar 15;180(6):289-91.
  33. Schulz K, Grimes D. Multiplicity in randomised trials II: subgroup and interim analyses. The Lancet. 2005;365(9471):1657-61.
  34. Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ. 2003 Jan 25;326(7382):219.
  35. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003 Sep 6;327(7414):557-60.
  36. Freemantle N. Interpreting the results of secondary end points and subgroup analyses in clinical trials: should we lock the crazy aunt in the attic? BMJ. 2001 Apr 21;322(7292):989-91.
  37. World Health Organization. WHO guide for standardisation of economic evaluations of immunisation programmes. 2008.
  38. Drummond M, Chevat C, Lothgren M. Do we fully understand the economic value of vaccines? Vaccine. 2007 Aug 10;25(32):5945-57.
  39. Beutels P, Van Doorslaer E, Van Damme P, Hall J. Methodological issues and new developments in the economic evaluation of vaccines. Expert review of vaccines. 2003 Oct;2(5):649-60.
  40. Kim SY, Goldie SJ. Cost-effectiveness analyses of vaccination programmes : a focused review of modelling approaches. PharmacoEconomics. 2008;26(3):191-215.
  41. Beutels P, Edmunds WJ, Antonanzas F, De Wit GA, Evans D, Feilden R, et al. Economic evaluation of vaccination programmes: a consensus statement focusing on viral hepatitis. PharmacoEconomics. 2002;20(1):1-7.
  42. Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical decision making : an international journal of the Society for Medical Decision Making. 1993 Oct-Dec;13(4):322-38.
  43. World Health Organization. Constitution of the World Health Organization. World Health Organization Handbook of basic documents. 5th ed ed. Geneva: Palais des Nations; 1952.
  44. Nord E. A review of synthetic health indicators. Background paper prepared for the OECD Directorate for Education, Employment, Labour, and Social Affairs. In: OECD Directorate for Education E, Labour, and Social Affairs, editor. 1997.
  45. Coons SJ, Rao S, Keininger DL, Hays RD. A comparative review of generic quality-of-life instruments. PharmacoEconomics. 2000 Jan;17(1):13-35.
  46. Devlin NJ, Hansen P, Kind P, Williams A. Logical inconsistencies in survey respondents' health state valuations -- a methodological challenge for estimating social tariffs. Health economics. 2003 Jul;12(7):529-44.
  47. Lamers LM, McDonnell J, Stalmeier PF, Krabbe PF, Busschbach JJ. The Dutch tariff: results and arguments for an effective design for national EQ-5D valuation studies. Health economics. 2006 Oct;15(10):1121-32.
  48. Patrick DL, Starks HE, Cain KC, Uhlmann RF, Pearlman RA. Measuring preferences for health states worse than death. Medical decision making : an international journal of the Society for Medical Decision Making. 1994 Jan-Mar;14(1):9-18.
  49. Perkins MR, Devlin NJ, Hansen P. The validity and reliability of EQ-5D health state valuations in a survey of Maori. Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation. 2004 Feb;13(1):271-4.
  50. Khanna D, Tsevat J. Health-related quality of life--an introduction. The American journal of managed care. 2007 Dec;13 Suppl 9:S218-23.
  51. Murray C. Rethinking DALYs. The global burden of disease A comprehensive assessment of mortality and disability from diseases, injuries, and risk factors in 1990 and projected to 2020: Harvard School of Public Health, on behalf of the World Health Organisation and the World Bank; 1996.
  52. Salomon JA, Vos T, Hogan DR, Gagnon M, Naghavi M, Mokdad A, et al. Common values in assessing health outcomes from disease and injury: disability weights measurement study for the Global Burden of Disease Study 2010. Lancet. 2012 Dec 15;380(9859):2129-43.
  53. Pharmaceutical Management Agency (PHARMAC). Pharmaceutical Schedule 2015. Available from: link).
  54. New Zealand Ministry of Health. New Zealand Health Survey: Annual update of key findings 2012/13. In: Health Mo, editor. Wellington: Ministry of Health; 2013.
  55. New Zealand Ministry of Health. New Zealand Health Statistics 2014. Available from: link).
  56. Xie F, Thumboo J, Fong KY, Lo NN, Yeo SJ, Yang KY, et al. A study on indirect and intangible costs for patients with knee osteoarthritis in Singapore. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2008 Mar;11 Suppl 1:S84-90.
  57. Metcalfe S, Grocott R. Comments on "Simoens, S. Health economic assessment: a methodological primer. Int. J. Environ. Res. Public Health 2009, 6, 2950-2966"-New Zealand in fact has no cost-effectiveness threshold. International journal of environmental research and public health. 2010 Apr;7(4):1831-4.
  58. Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Medical decision making : an international journal of the Society for Medical Decision Making. 1998 Apr-Jun;18(2 Suppl):S68-80.
  59. Lynd L. Quantitative Methods for Therapeutic Risk-Benefit Analysis. Issues Panel: Health outcomes approaches to risk-benefit analysis: how ready are they? 11th International Meetings of the International Society for Pharmacoeconomics and Outcomes Research; May 2006; Philadelphia, PA. 2006.
  60. Craig BA, Black MA. Incremental cost-effectiveness ratio and incremental net-health benefit: two sides of the same coin. Expert review of pharmacoeconomics & outcomes research. 2001 Oct;1(1):37-46.
  61. Zethraeus N, Johannesson M, Jonsson B, Lothgren M, Tambour M. Advantages of using the net-benefit approach for analysing uncertainty in economic evaluation studies. PharmacoEconomics. 2003;21(1):39-48.
  62. Garber A, Weinstein M, Torrance G, Kamlet M. Theoretical foundations of cost-effectiveness analysis. In: Gold M, Siegel J, Russell L, Weinstein M, editors. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996.
  63. Hughes DA, Bagust A, Haycox A, Walley T. Accounting for noncompliance in pharmacoeconomic evaluations. PharmacoEconomics. 2001;19(12):1185-97.
  64. Briggs A, Sculpher M, Buxton M. Uncertainty in the economic evaluation of health care technologies: the role of sensitivity analysis. Health economics. 1994 Mar-Apr;3(2):95-104.
  65. Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health economics. 2005 Apr;14(4):339-47.
  66. Agro KE, Bradley CA, Mittmann N, Iskedjian M, Ilersich AL, Einarson TR. Sensitivity analysis in health economic and pharmacoeconomic studies. An appraisal of the literature. PharmacoEconomics. 1997 Jan;11(1):75-88.
  67. Briggs AH. Handling uncertainty in cost-effectiveness models. PharmacoEconomics. 2000 May;17(5):479-500.
  68. Cohen BJ. Discounting in cost-utility analysis of healthcare interventions: reassessing current practice. PharmacoEconomics. 2003;21(2):75-87.
  69. Bos JM, Postma MJ, Annemans L. Discounting health effects in pharmacoeconomic evaluations: current controversies. PharmacoEconomics. 2005;23(7):639-49.


[1] Please note that, although not explicit on this diagram, the health needs of the family or whānau of the person receiving the treatment, and of wider society will be taken into consideration during our decision making process. This Factor is detailed in the Supporting Information that can be found on the PHARMAC website at

[2] Refer to Table 12: Reporting of Cost-Utility Analysis Results in Chapter 11 for further details on information to include in a CUA report when describing the disease, patient population and treatment options.

[3] Meta-analysis systematically combines the results of studies in order to draw overall conclusions about the efficacy and/or safety of the treatment.

[4] Observational studies register outcomes of groups of patients treated in ordinary clinical practice.

[5] The p value is the probability that an observed effect is due to sampling error; therefore, it provides a measure of the strength of an association. This section uses p values to notionally define statistical significance; however, it is noted that confidence intervals may better summarise the strength and precision of the effect estimate.

[6] Effect sizes with p values close to but not reaching statistical significance will be due to either one of two circumstances: (1) the effect is strong but the confidence interval is wide, because numbers of events, etc, are small; or (2) the effect is weaker but the confidence interval is narrower. In either case the p value being close to 0.05 means that the 95% confidence interval will only just include the value of 1.0 (ie a small but statistically significant chance that there is no effect). When deciding whether to still include such clinical events: (1) a strong effect will take precedence over a weaker effect; (2) a strong effect (with wide confidence limits) means the effect is likely to be clinically important, being limited by insufficient power (where ‘absence of evidence is not evidence of absence’) (18). Conversely, a weak effect with narrower confidence limits is unlikely to be clinically important (ie greater confidence but a negligible effect on outcomes).

[7] To help determine whether events are clinically significant, outcomes should be examined to determine whether their association with treatment is likely to be causal. Key criteria for determining causal associations include (19): temporality (ie the cause must precede the effect); strength of association; consistency between different populations and different study designs; and a dose-response relationship (ie increased exposure is associated with an increased biological effect).

[8] For composite endpoints to be valid, the results of the individual endpoints of composite measures reported by clinical trials should be reported (20). The numberof individual endpoints should be minimised to preferably no more than three or four (21). Component non-fatal endpoints should be measured appropriately, with the use of a blinded endpoints committee, a core laboratory, or both (21), and analysis of non-fatal events should take into account competing risks. For information on the assessment of composite outcomes, please refer to the PBAC Guidelines for preparing a major submission (22).

[9] Due to the differences in regulatory approval processes, this section applies mainly to medical devices.

[10] Patient subgroups may have different responses to treatment or magnitudes of benefit. These subgroups may be defined by age, gender, other demographic factors, disease-related factors (symptom complexes, severities), comorbidities, or intractability and factors affecting treatment effectiveness. The degree of breakdown depends upon the complexity of the targeting decisions to be made. Some situations will require many subgroups, others just the overall group.

[11] Relevant statistical tests of interaction include the chi-square test using the Q statistic in an individual trial or the Cochran Q statistic across the pooled result, and the I2 statistic with its 95% uncertainty interval.

[12] Statistical tests of interaction are preferred to individual tests within each subgroup – individual tests often overestimate the extent of true differences. (32)

[13] Subgroup treatment effects in a trial with no overall treatment effect are said to be usually superfluous subgroup salvages of otherwise indeterminate (negative) trials (33).

[14] DALYs are expressed in terms of years of life lost due to premature death and years lived with a disability of specific severity and duration.

[15] HYEs incorporate individual preference structures over a complete path of health states (rather than discrete health states).

[16] This included negative values for health states considered to be worse than death (47). Survey results indicated that respondents can and do evaluate some health states as worse than death, and the study authors recommended the systematic inclusion of these states to describe a more complete range of preference values (48).

[17] Logical inconsistency was defined as “when a state that ‘in logical terms’ is unambiguously less severe than another is assigned a lower value” (46).

< previous | table of contents