resume-parser accuracy benchmark

Question

I am currently using a resume parser API that parses resumes in bulk via a REST API call and provides output in JSON. What I am trying is to assess whether the extracted information (e.g. Education, Experience, Skills, Overview) fits with the original resume. Is there any Named Entity Recognition pipeline to test accuracy?

I've tried computing the Cosine similarity between the original CV and the blocks of text with a specific piece of information (e.g. Education, Experience, Skills, Overview) but I am looking for something more sophisticated and accurate to accomplish this. I have recently found http://www.davidsbatista.net/blog/2018/05/09/Named_Entity_Evaluation/, but still, I am seeking something that accounts for cv-specificity including sections and subsections.

Answer 1

To test the accuracy of a resume parser, you really need to build one yourself - and its not so easy. Note that NER tagging / evaluation is only a small step in the pipeline of the process. There are also lots of subtleties which may look insignificant to your scoring algorithm, but are important to humans - such as knowing that a certain date range applies to their time working at Microsoft, and not their time studying at College.

I think there are two approaches:

1. Just go through the output manually for 50 resumes and score the quality of the output manually.
2. Use another third party API (most of them will offer free trials), and programatically compare the outputs. (e.g. try https://affinda.com/resume-parser/, its one i helped build recently)