...
You want to create a project that enables you to label the sentiment in a given Tweet post as “Positive” Positive, “Negative” Negative, “Neutral” Neutral, or “Not Not Sure.”
To create this project, you must perform the following tasks:
...
Identify the sentiment type in a given Tweet as “Positive” Positive, “Negative” Negative, or “Neutral Neutral.”
In the cases where the sentiment cannot be determined by you, mark it as “Not Not Sure.”
Sample Input Data
You select sample tweets that you wish to annotate. You paste the text of the tweets into a Microsoft Excel sheet under a column labeled Tweet. You save the Microsoft Excel sheet as Tweet_Tagging_Input.xlsx.
...
You can follow the steps listed above to create your Microsoft Excel sheet; you can also download and use this file Sheet or download this file, unzip it, and use it in your project: Tweet_tagging_input.xlsx.
Each downloadable file is available in the ZIP format. To use it, unzip its contents.
Configuring Project Metadata
...
Taskmonk uses the project type that you specify to add input and/or output fields to projects as required. You can modify these later. In this instance, you selected Text-Based as the Project Type, and Taskmonk adds the following fields to the Task Design tab:
Input Field
Field Name: Text, Field Type: Text
Output Field
Field Name: Sentiment Type, Field Type: Radio Mandatory: False, Disabled: False, Customer Visible: False
...
select the field type for the input and output fields you’ve created under Task Design.
Click the Input Field tab to display the Input Field UI.By default, Taskmonk
sets the field type to Text. To edit thisClick the Create Input Field button.
To edit the input field details:
Click the Edit Field Name icon . and enter Tweet in the Name textbox appears.
Enter Tweet as Name.
Click the Update Field Name icon next to the Name textbox to save the changes.
Click the Field Type dropdown and set it to Text.
...
3. Either click the BROWSE INPUT FILE button to extract field names from the sample input file you’ve downloaded in the previous step, or
4. Click the CREATE INPUT FIELD button to create the input fields manually.
...
4. In this example, we’ve created two input fields, Tweet#and Tweet. Set Field Type to text, Mandatory to True, and Operational to True for both these fields.
...
Name: Provide the output field name.
...
(Optional) Visible Name: The name that appears in the output file.
...
Set Data Type: Select from the listed options how the output field options will appear. In this case, select Radio.
...
Possible Values: Type in comma-separated values that this output field can take. These values will be listed for radio button selection.
...
4. Select the All Levels check box to indicate that this field must be available to users at all execution levels, such as labeling, quality analysis, and so on.
5. Click Create to save the new output field and return to the Task Design > Output Fields tab.
...
6. Click Next to move to the next step, and click the Quality Workflow tab.
...
Creating Quality Workflows
The Quality Workflow tab enables you to specify how you want to ensure output quality. It also helps you create the execution levels required for your quality workflows. For example, you want to create the following levels for this project:
Annotator
Quality Analyst
Delivery Lead
In this instance, you want to enable a Majority-Vote workflow. Click the Execution Method field and select Majority-Vote from the drop-down list that appears.
...
2. By default, Taskmonk creates the Analyst role for you. Click the Update Execution Level icon. The Edit Execution Level modal appears.
...
3. Enter Annotator as the Execution Level Name.
4. You can skip all other fields here. Click Update to return to the Quality Workflow tab.
5. Click the Add Execution Level button on the right side of the page. The Add Execution Level modal appears.
6. Enter Quality Analyst in the Execution Level Name field.
7. You can skip all other fields here. Click Add.
Taskmonk adds the new execution level, closes the modal, and displays the updated Quality Workflow tab. Repeat the process to add all required execution levels.
...
8. Click Next to move to the next step, update User counts and Majority vote.
a. Set the User counts to 3 if every task has to be allocated to 3 different users. Majority vote 2 indicates that out of 3 users, if at least 2 of them provide the same response, the task completes at level 1. If all 3 users provide different responses, the task moves to level 2.
...
4. Click Add to add the selected users to their respective execution levels.
...
The users get added successfully to the Manage Users page.
...
Log in as Analyst and click the My Tasks icon at the top of the page . The to view the My Tasks page appears.
...
2. Click the Get Tasks button adjacent to the Twitter Post Analysis project. The labeling UI associated with this project appears.
...
4. Select the option for Sentiment Type and click Submit & Get Next Task.
5. The input fields details for the next task are displayed. Select the option Sentiment Type, and click Submit Task & Exit.
...
Repeat the step until all tasks are completed
For detailed information on working with a typical labeling UI, see Labeling Data.
Viewing Batch Status Report
Sign in as Manager, go Go to the Projects page and click Reports > View for the Twitter Post Analysis project. The Reports page appears.
Click Dataset Progress Reports to view the batch status report. This shows the total number of tasks pending and completed at each level for all batches. The Disagreement score indicates the total number of tasks without a majority at level 1. In other words, the total tasks which moved to level 2.
...
Downloadable Sample Files
...