Digital welfare dystopia reloading in New Zealand with new robo law
Image: Waldemar Brandt @waldemarbrandt67w via Unsplash
On 1 July a new law takes effect in New Zealand that will empower a bot to make decisions about welfare benefits for pensioners, the disabled and people with serious health conditions.
The Social Security (Modernisation) Amendment Bill was rushed through under urgency just before midnight on 29 May. It allows the Ministry of Social Development to use artificial intelligence to “make any decision, exercise any power, comply with any obligation, or take any other related action under any specified provision, with appropriate safeguards”.
That the bill was passed without Select Committee hearings, without public submissions, and with a redacted regulatory impact statement, raises numerous red flags.
The government’s position is that automation will be used only for simple, rules-based decisions, with human judgement retained where it counts. The ministry claims the changes will not involve generative AI, and that a narrower form of automated decision making has been in place since 2022. The government says safeguards remain in the legislation, including measures to manage bias, ensure transparency and maintain human oversight.
But opposition MPs have pointed out that automated social welfare decision making was tried and failed spectacularly in Australia. They point to the Royal Commission inquiry in 2023 into the Robodebt Scheme, which automated income-averaging to wrongly calculate and demand welfare debt repayments from benefit recipients, causing widespread harm before being scrapped.
The commission’s final report described it as a “crude and cruel mechanism” known from inception to be inconsistent with social security law. Its warnings evoked those of Philip Alston, UN Special Rapporteur, who had previously cautioned governments against “stumbling, zombie-like, into a digital welfare dystopia”.
The Royal Commission’s recommendations included transparency about AI decision making, making algorithms available for independent expert scrutiny, providing a review mechanism for those affected, and creating a monitoring body to audit for fairness, bias and usability. The Australian government accepted the recommendations, but they remain unlegislated three years later.
Critics of New Zealand’s new AI welfare law argue it should have stated explicitly what automated decision-making can and cannot be used for, rather than a broad general authorising provision, and called for stronger safeguards, proper consultation, and a modernised Privacy Act.
During a debate in the House, the Green Party’s Ricardo Menéndez March said it was “extremely concerning” that a bill which greatly expanded government powers was being passed without any scrutiny or consultation. “This is a carte blanche expansion to basically allow a robot, a machine, to have power of people’s lives.”
Others warned automation targets the most vulnerable and disconnected in society who need human contact rather than faster algorithms, and that promised safeguards could not be relied on in an unequal system where technology was hardly neutral.
