Configuring Project Quality Workflows

Owned by Kumar Luv (Unlicensed)
Last updated: Nov 08, 2021
5 min read

Specify the execution flows and process logic to be used in your project.

Overview

Once you have your project metadata and design in place, you must configure the execution method and workflow that you want to implement in your project. To do so, click the Quality Workflow tab in Project Settings.

You cannot add configure project workflows until you have configured the metadata and task design associated with your project.

 

 

What are execution methods?

Execution methods determine how Taskmonk uses input data during project execution. For example, the application can send input data to one analyst and then to users in the next execution level (editor or quality analyst); the application can also send input data simultaneously to multiple analysts and let the majority values persist.

Taskmonk offers the following execution methods:

  • Maker-Checker: If you select this execution method, Taskmonk sends input data to labeling analysts and, when the analyst completes labeling, routes the labeled input data to quality analysts, who approve or reject annotations (send them back for rework), but cannot edit them.

  • Maker-Editor: If you select this execution method, Taskmonk sends input data to labeling analysts and, when the analyst completes labeling, routes the labeled input data to editors, who can edit the labels created before approving them.

  • Majority Vote: If you select this execution method, Taskmonk sends input data to multiple labeling analysts simultaneously and, when all analysts complete labeling, passes the majority values as accepted labeled output.

What are execution levels?

In Taskmonk, each step in the process is implemented as an execution level. Thus, Level 1 in the process is always initial labeling. This level is created for you automatically when you create a project. You can update this level or add newer (higher) levels as required. You can also delete the first level created by default and configure your project workflow as required. Once you have created the execution levels required for your project, you can interconnect them by configuring the process logic that you want implemented in your project.

Managing Execution Levels

  1. Click Settings > Quality Workflow. The Quality Workflow tab appears.

  2. Click Execution. The Execution Level tab appears.

  3. Click the Execution Method and select the desired execution method. To know more, see What are execution (quality control) methods?.

  4. To add the next execution level, click Add Execution Level. Add execution level modal appears.

     

     

  5. Enter the Execution Level Name as required. The Level field cannot be edited as only the next level to the current highest level in the project can be added.

  6. Set Bulk Task Number to the maximum number of tasks that can be executed in bulk at the given execution level.

  7. Enable Skip Task to allow a user at the given execution level to skip tasks.

  8. Enable Reject Task to allow a user at the given execution level to reject tasks and return them to the previous level. This cannot be set for level 1.

  9. Click Add to add the execution level and return to the Execution Level tab.

  10. At any point during the process, click Cancel to return to the Execution Level tab without adding a new execution level.

  11. To update execution level details, click the Action > Update Execution Level icon corresponding to the level. Edit Execution Level modal appears.

  12. You can edit the execution level name, the number of bulk tasks allowed, and the ability to skip and reject tasks as required.

  13. Click Update to save the changes and return to the Execution Level tab.

  14. At any point during the process, click Cancel to return to the Execution Level tab without saving the changes.

  15. To delete a level, click the Action > Delete icon. Only the highest level present can be deleted from the project.

  16. Click Next to save execution levels and move to the Process Logic tab.

Managing Process Logic

Process logic determines how tasks flow from a given execution level to the next. You can decide the percentage of tasks that flow from one level to another. You can also decide which tasks should be assigned to the next level based on an attribute value or the result value. To do so:

  1. Click Settings > Quality Workflow > Process Logic. The Process Logic tab appears.

  2. To add process logic, click + Add Process Logic on the right-hand side of the page. Add Process Logic page appears.

     

     

  3. Select the values for From Level and To Level as required. Click the Add Rule icon to load the Add Rule UI.

  4. Click the Add New Rule icon. The Rule Type dropdown appears.

  5. Click the Rule Type and select the desired rule type. The following rule types are available:

    1. PercentageRule: To set the percentage of completed tasks that should move from a given level to the next. The percentage of tasks can be determined based either on the total number of tasks in the project or the total number of tasks executed by a user.

    2. UserAttributeRule: Determines which tasks to assign to the next level based on an attribute of the user who worked on the task being equal to/not equal to a predetermined value.

    3. ResultValueRule: Determines which tasks to assign to the next level based on an input/output field being equal to/not equal to a predetermined value.

    4. ResultMatchRule: Determines which tasks to assign to the next level based on an output field being equal to/not equal to another field value.

  6.  

    1. If Rule Type is set as Percentage Rule:

       

       

      1. Click the Percentage and select the desired percentage.

      2. Click the Percentage Scope and select the level at which the percentage should be calculated.

    2. If Rule Type is set as User Attribute Rule:

       

       

      1. Click Attribute Rule to select the desired attribute.

      2. Click Match Type to set the match operator as equal to or not equal to.

      3. Enter the Attribute Value as desired.

    3. If Rule Type is set as Result Value Rule:

       

       

      1. Click Field Name to select the desired attribute.

      2. Click Choose Operator to set the match operator as equal to or not equal to.

      3. Enter the Match Value as desired.

    4. If Rule Type is set as Result Match Rule:

      1. Click Field Name to select the desired field.

      2. Click Choose Operator to set the match operator as equal to or not equal to.

      3. Enter the Match Value as desired.

  7. Click Submit to add the configured rule.

  8. Define additional rules as required.

  9. To edit an existing rule, click the Action > Edit icon corresponding to the required rule in the Process Logic tab. Add Rule UI appears.

  10. To delete an existing rule, click the Action > Delete icon corresponding to the required rule in the Process Logic tab.

Related Links

 

© 2020 Taskmonk Technology Pvt. Ltd. All Rights Reserved .