Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pos: Tagger performance is insufficient on build? #743

Open
chrisoutwright opened this issue Dec 25, 2020 · 0 comments
Open

Pos: Tagger performance is insufficient on build? #743

chrisoutwright opened this issue Dec 25, 2020 · 0 comments

Comments

@chrisoutwright
Copy link

chrisoutwright commented Dec 25, 2020

The following test is performed after build for POSAnnotator();
:

    @Test
    public void testAnnotatorDiff() {
        POSAnnotator annotator = new POSAnnotator();

        TextAnnotation record =
                BasicTextAnnotationBuilder.createTextAnnotationFromTokens(refTokens);
        try {
            annotator.getView(record);
        } catch (AnnotatorException e) {
            fail("AnnotatorException thrown!\n" + e.getMessage());
        }

        TokenLabelView view = (TokenLabelView) record.getView(ViewNames.POS);
        if (refTags.size() != view.getNumberOfConstituents()) {
            fail("Number of tokens tagged in annotator does not match actual number of tokens!");
        }
        int correctCounter = 0;
        for (int i = 0; i < refTags.size(); i++) {
            if (view.getLabel(i).equals(refTags.get(i))) {
                correctCounter++;
            }
        }
        double result = ((double) correctCounter) / refTags.size();

        if (result < thresholdAcc) {
            fail("Tagger performance is insufficient: " + "\nProduced: " + result + "\nExpected: "
                    + thresholdAcc);
        }

    }

I get the following error:

java.lang.AssertionError: Tagger performance is insufficient: 
Produced: 0.3541666666666667
Expected: 0.95

I printed out all tags and corresponding pos and it seems to fail in testDiff() at these only:
NN VBD shot
JJR RBR more
Why is the testAnnotatorDiff() failing? There are a lot of (NN) for getLabel(i)! What does testAnnotatorDiff() do differently?
I don't get why the annotator.getView(record) is not assigned to any View object ... I just wanted to get the pipeline working but the required components need to build first, which is failing in my case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant