In this example, you create a video annotation project to identify and label roads and signboards in a video recording.

Project Overview

You want to create a project that enables you to identify and label roads and signboards in a video recording.

To create this project, you must perform the following tasks:

1. List out your project requirements.

2. Identify sample videos that you can use for labelling.

3. Configure project metadata.

4. Manage your project input and output fields.

5. Create the workflow that you want to implement in your project.

6. Add users to your project and assign them project roles.

7. Add a dataset to the project.

8. Start labelling input video.

9. View batch status reports.

This document explains how you can perform each of the tasks listed above. Specific sections in this document also contain sample data that you can use to easily create and implement this project in Taskmonk.

Listing Project Requirements

In this project, you want to:

• Identify and label roads and signboards in a video recording.

Sample Input Data

You select the stock video recordings that you wish to annotate. You paste the URLs of the recording into a Microsoft Excel sheet under columns labelled MediaUrl. You save the Microsoft Excel sheet as Video_Annotation_Input.xlsx on your hard drive.

Open

Download Source File

You can follow the steps listed above to create your Microsoft Excel sheet; you can also download and use this file in your project:

Open

Configuring Project Metadata

Project Metadata is the first tab that appears when you create a project. The Project Metadata tab enables you to provide basic information, such as the name, process, and project type, associated with your project. You can also upload any documentation that you may want to add to your project.

1. To create the project, click the Create Project floating button on the left side of the Projects page.
   
   

Open

The Project Metadata tab associated with your new project appears.

2. Enter New Video as the Project Name and test as the Process Name.

3. Click Project Type > Annotation.

4. Click Next.
   The Documents sub-tab appears.

5. You can upload documents associated with the project if required. This is an optional step, and you can skip it for now.
   Click Next.

The Task Design tab appears. Use this tab to manage your project input and output fields.

Managing Project Input and Output Fields

Taskmonk uses the project type that you specify to add input and/or output fields to projects as required. You can modify these later. In this instance, you selected Annotation as the Project Type, and Taskmonk adds the following fields to the Task Design tab:

• Input Field

  • Field Name: MediaUrl, Field Type: Image

• Output Field

  • Field Name: Annotations, Field Type: Annotation, Mandatory: False, Disabled: False, Customer Visible: True

  • Field Name: Classes, Field Type: Class, Mandatory: False, Disabled: False, Customer Visible: True

  • Field Name: Scenes, Field Type: Scene, Mandatory: False, Disabled: False, Customer Visible: True

  • Field Name: Segments, Field Type: segment, Mandatory: False, Disabled: False, Customer Visible: True

Updating Input Field Details

1. Click the Input Field tab to display the Input Field UI.

By default, Taskmonk sets the field type to Image. To edit this, select Field Type > Video.

Creating Output Field Details

Click the Output Field tab to display the Output Field UI.

Video annotation projects need Scenes and Segments.

To add Scenes and Segments output fields to your project:

To the Classes field, you wish to add two possible values: Road and Signboard. To do so, click Possible Values > Edit Classes. The Manage Possible Values page appears.

1. Enter Road as the Class.

2. Select Color as required.

3. Enter Road as the Class Label.

4. Click Save and Add Another to add the value.

5. Enter Signboard as the Field Name.

6. Select Color as required.

7. Click Save and Add Another to add the value.

8. Click Save & Close to save changes to the field and return to the Output Field tab.

Repeat the same steps for Scenes.

Click Next to move to the next step, Quality Workflows.

To the Classes field, you wish to add two possible values: Road and Signboard. To do so, click Possible Values > Edit Classes. The Manage Possible Values page appears.

Creating Quality Workflows

The Quality Workflow tab enables you to specify how you want to ensure output quality. It also helps you create the execution levels required for your quality workflows. For example, you want to create the following levels for this project:

• Annotator

• Quality Analyst

• Delivery Lead

1. In this instance, you want to enable a Maker-Editor workflow. Click the Execution Method field and select Maker-Editor from the drop-down list that appears.

2. By default, Taskmonk creates the Analyst role for you. Click the Update Execution Level icon. The Edit Execution Level modal appears.

3. Enter Annotator as the Execution Level Name.

4. You can skip all other fields here. Click Update to return to the Quality Workflow tab.

5. Click the Add Execution Level button on the right side of the page. The Add Execution Level modal appears.

6. Enter Quality Analyst in the Execution Level Name field.

7. You can skip all other fields here. Click Add.
   Taskmonk adds the new execution level, closes the modal, and displays the updated Quality Workflow tab. Repeat the process to add all required execution levels.

8. Click Next to move to the next step, Process Logic.

Creating Process Logic

Quality Workflow > Process Logic tab allows you to decide the logic based on which datasets are moved from one execution level to the next. For the current project, you want to enable the following rules:

• 80% of the project datasets move from level 1 to level 2.

• 65% of the project datasets move from level 2 to level 3.

To do so:

1. Click the +Add Process Logic button on the right-hand side of the page. Add process logic page appears.

2. Click the From Level dropdown and select L1 [Annotator].

3. Click the To Level dropdown and select L2 [Quality Analyst].

4. Click the Add Rule icon to load the add rule UI.

5. Click the Add New Rule icon. The Rule Type dropdown appears.

6. Click the Rule Type dropdown and select PercentageRule.

7. Click the Percentage dropdown and select 80%.

8. Click Submit to save the rule.

9. Repeat the above step to add rules for each level change.

Managing Users and Role

You must now add users to your project and assign the execution levels you just created to them.

1. Click the Users tab just above the Quality Workflow tab.
   The Users > Manage Users tab appears.

2. Click the Add button in the top-right section of the tab.
   The Select Users tab appears.

3. Corresponding to each execution level, click the Select Users field and select the desired user from the drop-down list that appears.

4. Click Add to add the selected users to the project.

5. Close the modal.
   The Manage Users tab reloads to display the updated user details.

Managing Project Datasets

Your project is now configured. Congratulations!

Before you can start transcription and labelling, you must upload the input data containing the audio to be labelled.

1. Click the Datasets tab. The Datasets page appears. Use this page to manage datasets for your project.

2. Taskmonk organizes datasets into batches to simplify management and tracking. To add a new dataset, click Add Batch on the right side of the page. The Add Batch modal appears.

3. Enter Batch 1 as the name for the batch that you want to import in the Add New Batch field. You can ignore the other fields.

4. Click Submit. This creates a new batch of data for your project and adds it to the Pending tab of the Datasets page. You can now upload datasets into the batch, as required.

5. To add a dataset to the batch, click the Import button under the Tasks(Import/Export) column. The Import Task modal appears.

6. Click Choose Files, select the sample input file from your computer and click Import.

7. Once the dataset is imported, click Close to exit the modal.

Labeling Video Using Taskmonk

Your project is now ready for work.

1. Log in as a Transcription Analyst and click the My Tasks icon at the top of the page. The Tasks page appears.

2. Click the Get Tasks button. The labelling UI associated with this project appears.

You can see the following project details in the labelling UI:

• Batch Name (Batch 1) in the top-left section of the page.

• A video player with playback controls and a timeline.

• Toolbox containing various tools for creating annotations.

• Scenes to classify scenes in a video.

• Segments - Utilise this feature to further break down the scenes.

• Classes and objects to classify elements in the video.

• Audio Waveform for reference.
  For detailed information on working with a typical labelling UI, see Labeling Data.

Viewing Batch Status Report

1. Go to the Projects page and click Reports > View for the POS Tagging project. The Reports page appears.

2. Click on Dataset Progress Reports to view the batch status report. This shows the total number of tasks pending and completed at each level for all batches.