Clean, transform and filter data with Dova visual mappings

The operational problem

A file may contain extra spaces, mismatched column names or rows that do not belong in the process. Fixing every delivery manually makes it difficult to repeat the same rules and explain why a record was excluded.

Operation order matters too: comparing a text amount before converting it may produce a different selection from the numeric rule you intended. You need a design that makes each stage explicit.

How Dova helps

Dova’s Mapping editor connects source, transformation and destination components on a canvas. The execution engine implements text cleaning, field selection and renaming, type conversion and row filtering, alongside other documented transforms.

Configure each component’s fields and options, validate the design and publish a version to execute through a Session. This keeps rules reviewable and the structure reusable, separate from specific file paths. Compare the result with an independently defined expected output.

How it works

1. Define the expected output

   Identify fields, types and inclusion criteria. Prepare synthetic rows covering valid records, empty values and values exactly on the filter boundary.

2. Connect sources and transforms

   Add Flat File input and the components you need: for example Clean text for a name, Convert type for an amount and Row filter for selection.

3. Configure explicit rules

   Choose the text fields and cleaning operations; define the target type and conversion-error behavior. Use an available filter operator and a value appropriate for the type.

4. Validate and publish

   Review ports, columns and operation order. Save the draft, publish the Mapping and bind it to a Session with laboratory paths and a compatible worker.

5. Compare the records

   Execute a test Workflow and inspect Monitor. Compare included and excluded rows, types and values; adjust the rule before using it in an operational flow.

What you need

• An input sample and cleaning, conversion or filtering rules agreed with the data owner.
• Permissions to edit and publish the Mapping, plus a compatible Session, Workflow and worker.
• An independent test destination and an expected output you can verify beyond the execution status.

Limits to consider

• Row filter uses the operators available in its configuration; it is not an open SQL language and does not execute arbitrary instructions.
• Calculated field substitutes values into templates. It should not be treated as a general arithmetic formula editor.
• Filtering discards rows according to the rule. It does not create a rejected-record file by itself; quality checks do not imply automatic quarantine either.

Demonstration example

id,name,status
1,  demo one  ,active
2,demo two,inactive