An empirical evaluation of using ChatGPT to summarize disputes for recommending similar labor and employment cases in Chinese
This work addresses a domain-specific problem for legal professionals in China by improving case recommendation systems, but it is incremental as it builds on existing methods with new data sources.
The paper tackled the problem of recommending similar labor and employment cases in Chinese by developing a hybrid mechanism that uses itemized disputes for classification, and found that using GPT-4-generated disputes led to better results than court-prepared or GPT-3.5-generated ones, though performance with ChatGPT was satisfactory.
We present a hybrid mechanism for recommending similar cases of labor and employment litigations. The classifier determines the similarity based on the itemized disputes of the two cases, that the courts prepared. We cluster the disputes, compute the cosine similarity between the disputes, and use the results as the features for the classification tasks. Experimental results indicate that this hybrid approach outperformed our previous system, which considered only the information about the clusters of the disputes. We replaced the disputes that were prepared by the courts with the itemized disputes that were generated by GPT-3.5 and GPT-4, and repeated the same experiments. Using the disputes generated by GPT-4 led to better results. Although our classifier did not perform as well when using the disputes that the ChatGPT generated, the results were satisfactory. Hence, we hope that the future large-language models will become practically useful.