An excellent article from Eva Cox via Women for wik

Crikey Thursday, 10 February 2011 / Whatever happened to evidence-based policy making?
by Eva Cox

Prime Minister Gillard in parliament yesterday:

Because I believe in tackling the big challenges in the national
interest. I see Closing the Gap as a way of understanding the problems.

It is evidence-based, accountable and transparent. It tells us what
needs to be done first and fastest and builds a methodical approach. It
allows us to build consensus in support of specific progress, instead of
debating abstract ideas. To do what we can, with what we have, where we are.

Because I believe Australians judge Governments on delivery . I see
Closing the Gap as a way of working on the solutions.

It is a way of making specific, measurable progress. It is practical
and cumulative. It gives us new information which means we can invest where
investment will make the greatest difference. Information which means we can
be sure that the Government is meeting its responsibilities.

Worthy sentiments. If only they stacked up. Take the New Income
Management program. The federal government is adding another serious
question to its social and financial policy competence by informing the
public that it is proceeding with the promised evaluation of NIM. The
initial form of income management (IM) was a Howard initiative which was
justified as part of an “emergency” measure presumed to protect Aboriginal
children.

The ALP government extended the IM program as a core part of the NT
Intervention, thereby quarantining 50% of the benefit and pension incomes of
all residents in 73 designated Aboriginal communities. However, when the UN
was highly critical of the program’s suspension of Racial Discrimination Act
as it breached Australia’s obligations under it, the government changed
tack.

To be able to withdraw the suspension, Macklin decided to de-racialise
the program by passing legislation in June 2010 that allowed the Australian
government to expand the program. Now they can compulsorily quarantine the
incomes of anyone on benefits in certain time or age based categories (but
not more respectable pensions) in any declared area.

This change was pursued despite much solid evidence and presentations
to a Senate Community Affairs Committee that the program had not proved its
value in its previous three year existence. The government’s case was that
this expansion was part of an evidence based broader welfare strategy to
address social exclusion.

Despite government claimed “evidence” being refuted by a wide range of
academic and welfare groups and the government’s proposal for extension
being supported by only three of nearly a hundred submissions, the
government majority (and the Abbott Opposition in Parliament) backed the
legislation. The Majority Report did suggest that the lack of evidence
needed to be addressed before any further extension of IM to other areas and
groups. Ergo the evaluation of the proposed NT extension that has finally
emerged.

The participation categories that remain eligible totalled 11,564 in
June 2010, and another 9510 were on pension type payments not covered by
compulsory payments under the new Act.

Unfortunately for the disadvantaged population in the rest of
Australia this expansive and expensive evaluation is unlikely to protect
their current payments systems from changes, as evidence from the three-year
evaluation is unlikely to seriously affect or inform this government’s
policy actions.

While the media release may imply otherwise, the intentions of the
government are clear in its terms of reference on p3 of the executive
summary. The terms of reference for developing the evaluation framework are
that the evaluation:

a.. be completed by December 2014
b.. provide information on the implementation of the NIM in the
Northern Territory by the end of 2011 in order to inform decisions about an
expansion of the model beyond the Northern Territory
c.. result in data being collected that can be used to evaluate
short, medium and, where possible, longer-term impacts/outcomes of new
income management,
d.. include a set of ethics guidelines and an ethical clearance
strategy relevant to this evaluation project.

The above terms of reference clearly suggest that the government will
decide early in 2012 to extend the scope of the involuntary program to
beneficiaries outside the NT. The timeline means this will happen despite
little or no evidence by then of whether there are actual benefits from
program for recipients.

This change fits with the PM’s already stated intention of clamping
down on welfare and pressuring people into the paid work and a shift from
welfare rights to massively conditional welfare payments.

The change will happen despite serious questions on whether it worked
during the Intervention and a reluctance to wait and see whether evidence
from this evaluation will find indications of effectiveness and cost
benefits, in particular whether the substantial extra admin costs of around
$100M per annum could be better spent on other services for this group.

The evaluation document does illustrate both the difficulties of
finding evidence from any other examples of such programs and of evaluating
this particular program. These problems arise from the program’s diverse
origins and add-ons by government over the last three years. Given that the
initial form of income management was justified as part of an “emergency”
measure presumed to protect Aboriginal children, it was not opposed by the
then Rudd Opposition and most of the welfare sector, just some Aboriginal
groups.

All of which means no evidence of its value in this area has ever been
offered, in fact a recent report to the NT government on showed child
protection had deteriorated over the last three years, and failed to mention
the intervention.

This is now nearly nine months after the government declaring certain
benefit recipients in the whole NT as subject to the New Income Management,
as well as transferring most existing recipients to the ‘new’ scheme. This
version includes a few new bribes for those who voluntarily sign onto income
management and a complex, quite difficult exit system for those who may
consider they do not need to be managed.

This includes compulsory budgeting workshops and financial literacy
tests. There is also evidence that Centrelink is trying really hard to
retain those no longer covered as voluntary clients as outlined by Paddy
Gibson last year.

There had been no sign of the proposed evaluation till last week when
the Minister and FaHCSIA media release emerged and stated:

The Australian Government is today releasing the framework for the
independent evaluation of the new model of non-discriminatory income
management. The Government has now rolled out the new model of
non-discriminatory income management across the Northern Territory,
including child protection income management. A voluntary income management
and child protection income management pilot operates in Western Australia.

Future roll-out of the new model of income management beyond the
Northern Territory will be informed by evidence gained from the independent
evaluation.

The evaluation will include analysis of existing data on income
management, as well as surveys of child protection staff, financial literacy
service providers, and retailers. There will also be interviews and focus
groups with people on income management.

The rest of the first stages of this evaluation (2011) mainly cover
views on the processes of implementing the program by everyone but the
recipients. While the release covers an extensive evaluation to report
finally in 2014, as indicated above, the government will not wait for the
results.

There are other questions on the value of the evaluation document. One
is whether much of the data from the NT would apply in the very different
locations and populations in the rest of the states and territories. However
a major difficulty is the structure of the various bits of the program which
create complications in any evaluation of this program. To sum up the
various difficulties as derived from the document:

a.. There is no baseline data because the program has already been
implemented but in different places, times and ways.
b.. There are multitude of “populations” within those on NIM that
have different entry points, needs and possible exits:
1.. These include the majority of compulsory recipients whose
criteria for entry have no basis in their income management deficits but
they just qualify because they are on certain benefits.
2.. Another group “volunteers” to sign up and there is current
pressure and bribes in WA and elsewhere to add to these.
3.. Some are included because they are deemed “vulnerable” by
Centrelink social workers because of certain very broad criteria and
compulsorily placed on IM.
4.. Some are referred as parents who are reported to child
protection authorities because they are neglecting their children or
appropriate care.
5.. There are unconfirmed reports that some refugee families are
being targeted because they are sending money to families overseas.
6.. There are also a limited number of Cape York families and
maybe a few others who are disciplined by being put on reduced payments
because of failure to send children to school
7.. The WA programs and many others are embedded with other
support services which are patchy and not available elsewhere
8.. There are massive language and cultural issues in NT and some
other isolated communities which make interviewing and informed consent
difficult and raise major ethical issues.
c.. The difficulties of defining common ‘problems’ in such diverse
groups, particularly as many issues do not relate clearly to their financial
management
d.. This means it is very difficult to set outputs and outcomes to
measure because the common problems may not exist (see below)
e.. The problems many would manifest in the compulsory category of
benefit groups may not relate to personal difficulties in managing
financially, but reflect the inadequacy of benefits that may not meet basic
costs in urban settings.
f.. There are indications that criteria for exemptions set
impossibly high standards which will discourage people from applying
g.. As most of these groups are also involved in other programs,
often as part of the above categorisations, can research tell what changes
are causal?
h.. As some have been on already for three years plus and some only
a few weeks, how and when can relative changes be measured?
i.. What culturally appropriate measures are there to assess
possible negative effects of shame and anger at being targeted in this way?
These show how the particularities of the NT and origins of the
program in the Intervention make it unlikely that any findings would be able
to be applied widely.

The evaluation document does recognise many of these problems, but
makes some comments about “triangulating” data to make up for the gaps.
However, the authors fail to critically question whether there is value in
undertaking such a complex project with such possible flaws. They do not
raise questions on the misuse of the evidence for the policy expansion
decision nor question whether this process is primarily a post hoc searching
for validating evidence.

It is not clear in the evaluation document whether there may be other
benefits in pursuing research on this type of welfare change. The question
is why has Macklin funded this complex and costly evaluation?

The answers only make sense if we connect this project to other
aspects of the current government archaic models of delivering current
social policy. The driving principles appear to be beliefs that “social
inclusion” can be achieved by fitting the excluded into the interstices of
current social systems. The basic assumption is that the excluded are flawed
not the society.

Ergo, those on welfare for a long time must have serious personal
issues, which inhibit their entry to paid work. The government then assumes
the right, or maybe the moral duty, to coerce their compliance to models of
better behaviour which will fix the problems.

In this case, the government (Macklin?) assumes regularising
recipients’ money practices will create some order in their lives, and
therefore result in better parenting, higher workforce participation and
other desirable ends. These beliefs are spelt out in the paper with examples
of ‘conditional welfare payments. These types of changes fail to answer the
question of causation and ignore entirely the caveat expressed by the
evaluation schema authors:

Many who have written about conditionality, whether in its favour or
not, conclude that conditionality can be philosophically and morally
justified provided that considerable care is taken to avoid burdening those
people who are already unjustly disadvantaged (Deacon 2004). P51.

Most of those on benefits, such as Aboriginal people are already
unjustly disadvantaged by systems and prejudice.

The evaluation paper academics have included a literature review with
presumably, a wide search for examples of similar schemes as exemplars for
the current process. Those reported on do not offer much in direct evidence
that there are similar schemes that have worked. The listed examples do not
offer any serious evidence that conditional welfare works well as a national
policy model in developed countries.

Many of the thirteen quoted examples are relatively small programs,
often not continuing and set up in very different circumstances.

There are three USA programs, two of which are Food Stamps and TANF,
which are the only national ones in developed countries. One a private NY
program that was short term, and there is another short term private
Canadian program. The other nine programs cited are all in developing
countries: Mexico, Malawi, and Brazil. India, Bangladesh, Columbia, Honduras
and Guatemala, dealing with very different problems and cultures, let alone
economic systems.

These are odd countries for Australia to search for examples. The
designs and problems faced, e.g. birthrates and malnutrition, are not
necessarily related to Australia. So, apart from some questionable benefits
from the named US programs, there are no developed comparable welfare states
that have adopted such massive welfare system changes.

Given these constraints and problems, I wonder why reputable academic
institutions such as SPRC, AIFS and the ANU would engage in this project and
risk legitimating bad decision making and further examples of evidence being
ignored? Could it just be the temptation of the money for a research
starved institution? Are there too many self-funded research units at
universities being government servants? I did originally write tarts but
many s-x workers may have better ethics.

Advertisements