Economic malaise or shocks, such as those caused by the COVID-19 pandemic, can result in job losses and individuals turning to government income support. Some may find themselves relying on government transfers for longer than anticipated, leading to prolonged (and unintended) spells of welfare receipt. Extended periods of welfare can have adverse effects on recipients and create significant expenditures for governments.
Our paper, published in the Oxford Bulletin of Economics and Statistics, demonstrates how machine learning algorithms serve as a valuable tool for understanding the risk of long-term welfare receipt. The combination of high-quality big data and machine learning methods allows researchers to provide more accurate predictions than commonly used benchmark models.
Machine learning can be used in combination with large administrative datasets
Machine learning has been applied in various ways within the welfare system to improve program effectiveness. For example, Sweden’s social services have used it since 2015 to streamline the processing of social assistance applications. In other areas, machine learning has aided judges in improving bail-granting decisions, assisted schools in identifying students at risk of dropping out, and supported surgeons in screening patients for hip-replacement surgery.
Machine learning provides policymakers with a data-driven approach to target scarce resources by processing complex datasets, identifying patterns, and prioritising individuals or areas in need.
In this study, we predict who is at risk of long-term welfare receipt using data from the full population of social security enrolees in Australia. This dataset comprises daily records of income support receipt behaviours for millions of individuals and their household members, along with various demographic and socioeconomic data.
In the initial setup, we started with 1,800 characteristics of the individual. The extensive scale and comprehensive nature of the dataset make it well-suited for machine learning applications, allowing algorithms to excel in detecting intricate data patterns and discovering key predictors.
Given the vast amount of data available, the utilisation of machine learning algorithms becomes essential to safeguard against overfitting. This occurs when a model can identify peculiar trends in the training data but struggles to derive general patterns, resulting in subpar performance on new data.
Using procedures to safeguard against overfitting, such as cross-validation, and the richness of the data, we show that machine learning algorithms can significantly improve predictive accuracy compared to simpler heuristic models or early warning systems currently in use. Specifically, our machine learning algorithms predicts the proportion of time individuals are on income support in the subsequent 4 years with greater accuracy, by a magnitude of at least 22% (14 percentage points increase in the R-squared), compared to baseline models.
The machine learning algorithms we employed also detected subtle patterns in the data – and characteristics of individuals – not commonly associated with long-term welfare receipt. For example, it detected factors such as the extent of annual income variability, the frequency of residential relocations, and the probability of failing to meet mutual obligation criteria. These factors were only clear in their connection to prolonged welfare receipt through iterative training and learning from the available data using machine learning models.
Remaining challenges and next steps
Despite their growing popularity, there is still a considerable degree of scepticism about the impact of adopting automated systems due to concerns about accuracy and bias reinforcement. The Online Compliance Intervention (OCI), colloquially known as Robodebt, designed to improve efficiency by automating the Centrelink debt recovery process, serves as yet another illustration of the importance of exercising caution when dealing with automated systems. Establishing a system to monitor and audit automated decision-making, as was recommended by the Robodebt Royal Commission, is a first step towards minimising potential harm from automated systems.
This is why we do not advocate for our algorithms to replace human expertise but rather to complement it. For example, caseworkers could focus their attention and time on providing personalised service and targeting appropriate support to individuals that the algorithm identifies as most at risk.
Alternatively, caseworkers could tailor interventions based on risk scores to improve the efficiency and effectiveness of social services. For individuals with low-risk scores, the focus may be on light-touch interventions, such as online resources or periodic check-ins. Those with moderate risk scores might receive more personalised support, including job training or counselling. High-risk individuals may require intensive case management, access to mental health services, or housing assistance.
As machine learning systemises the prediction process, it may reduce both conscious and unconscious biases common in human decision-making, as well as minimise the impacts of caseworker absence on an individual’s total duration on welfare receipt. Importantly, the approach is also relatively low-cost to implement since it exploits administrative data already available to practitioners.
However, identifying individuals who are at risk is only the first step. Policymakers seeking to assist eligible individuals in transitioning out of the welfare system must also understand the effectiveness of interventions and the specific individuals for whom these interventions are most likely to yield positive results. Prediction alone cannot answer these questions; causal estimations, such as those obtained from randomised controlled trials, are required here, ideally combined with machine learning to identify the sub-population of interest.
But why must individuals be treated as atomistic? In fact, we don’t. People are means tested on family income yet the tax system ignores voluntary income support within and between families. A rational tax transfer system would recognise income transfers to spouses, children and relatives and tax and means test accordingly on the basis of the actual enjoyment of the income. The growth of the burden of welfare spending has gone hand in hand over the last 60 years with dependant shedding by families who used to look after themselves and do for love what money can’t buy.