AI, Vol. 4, Pages 128-171: Augmented Behavioral Annotation Tools, with Application to Multimodal Datasets and Models: A Systematic Review

Annotation tools are an essential component in the creation of datasets for machine learning purposes. Annotation tools have evolved greatly since the turn of the century, and now commonly include collaborative features to divide labor efficiently, as well as automation employed to amplify human efforts. Recent developments in machine learning models, such as Transformers, allow for training upon very large and sophisticated multimodal datasets and enable generalization across domains of knowledge. These models also herald an increasing emphasis on prompt engineering to provide qualitative fine-tuning upon the model itself, adding a novel emerging layer of direct machine learning annotation. These capabilities enable machine intelligence to recognize, predict, and emulate human behavior with much greater accuracy and nuance, a noted shortfall of which have contributed to algorithmic injustice in previous techniques. However, the scale and complexity of training data required for multimodal models presents engineering challenges. Best practices for conducting annotation for large multimodal models in the most safe and ethical, yet efficient, manner have not been established. This paper presents a systematic literature review of crowd and machine learning augmented behavioral annotation methods to distill practices that may have value in multimodal implementations, cross-correlated across disciplines. Research questions were defined to provide an overview of the evolution of augmented behavioral annotation tools in the past, in relation to the present state of the art. (Contains five figures and four tables).

Apr 23, 2024 - 22:50
 0  150
AI, Vol. 4, Pages 128-171: Augmented Behavioral Annotation Tools, with Application to Multimodal Datasets and Models: A Systematic Review

AI, Vol. 4, Pages 128-171: Augmented Behavioral Annotation Tools, with Application to Multimodal Datasets and Models: A Systematic Review

AI doi: 10.3390/ai4010007

Authors: Eleanor Watson Thiago Viana Shujun Zhang

Annotation tools are an essential component in the creation of datasets for machine learning purposes. Annotation tools have evolved greatly since the turn of the century, and now commonly include collaborative features to divide labor efficiently, as well as automation employed to amplify human efforts. Recent developments in machine learning models, such as Transformers, allow for training upon very large and sophisticated multimodal datasets and enable generalization across domains of knowledge. These models also herald an increasing emphasis on prompt engineering to provide qualitative fine-tuning upon the model itself, adding a novel emerging layer of direct machine learning annotation. These capabilities enable machine intelligence to recognize, predict, and emulate human behavior with much greater accuracy and nuance, a noted shortfall of which have contributed to algorithmic injustice in previous techniques. However, the scale and complexity of training data required for multimodal models presents engineering challenges. Best practices for conducting annotation for large multimodal models in the most safe and ethical, yet efficient, manner have not been established. This paper presents a systematic literature review of crowd and machine learning augmented behavioral annotation methods to distill practices that may have value in multimodal implementations, cross-correlated across disciplines. Research questions were defined to provide an overview of the evolution of augmented behavioral annotation tools in the past, in relation to the present state of the art. (Contains five figures and four tables).

What's Your Reaction?

like

dislike

love

funny

angry

sad

wow

Lema Kefyalew Studying a Ph.D. in Civil Engineering (Hydraulic Engineering). Studying Master of Project Management (MPM). Graduated with MSc in Civil Engineering (Hydraulic Engineering). Followed a 12 month Software Engineering program at ALX_Africa (Holberton school). Graduated with BA degree in Management. Graduated with BSc degree in Civil Engineering.