Variational Bayes survival analysis for unemployment modelling
This work provides a method for predicting individual employment probabilities over time, which is useful for public employment services and potentially other domains with censored, high-cardinality categorical data.
This paper develops a variational Bayes survival analysis model with a deep artificial neural network as a non-linear hazard function to predict the probability of a job seeker finding a job over time. The model was evaluated on Slovenian unemployment data from 2011-2020, determining individual employment probabilities.
Mathematical modelling of unemployment dynamics attempts to predict the probability of a job seeker finding a job as a function of time. This is typically achieved by using information in unemployment records. These records are right censored, making survival analysis a suitable approach for parameter estimation. The proposed model uses a deep artificial neural network (ANN) as a non-linear hazard function. Through embedding, high-cardinality categorical features are analysed efficiently. The posterior distribution of the ANN parameters are estimated using a variational Bayes method. The model is evaluated on a time-to-employment data set spanning from 2011 to 2020 provided by the Slovenian public employment service. It is used to determine the employment probability over time for each individual on the record. Similar models could be applied to other questions with multi-dimensional, high-cardinality categorical data including censored records. Such data is often encountered in personal records, for example in medical records.