Skip to content

AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging (ACL 2019 Demo)

Notifications You must be signed in to change notification settings

INK-USC/AlpacaTag

Repository files navigation

AlpacaTag

AlpacaTag is an active learning-based crowd annotation framework for sequence tagging, such as named-entity recognition (NER).

Website      Documenations      Paper      Poster

The UI framework of AlpacaTag is based on the awesome work of Doccano, while the distinctive advantages of AlpacaTag are three-fold:

  • Active intelligent recommendation: dynamically suggesting annotations and sampling the most informative unlabeled instances with a back-end active learned model.

  • Automatic crowd consolidation: enhancing real-time inter-annotator agreement by merging inconsistent labels from multiple annotators.

  • Real-time model deployment: users can deploy their models in downstream systems while new annotations are being made.

  • Overall Workflow

Documentations

Citation

@InProceedings{acl19alpaca, 
     author = {Bill Yuchen Lin and Dongho Lee and Frank F. Xu and Ouyu Lan and Xiang Ren}, 
     title = {AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging.}, 
     booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Demo Track},
     year = {2019} 
}

Demo

Back-end Model APIs

Crowd Consolidation

Performance Study