Showing posts with label Knime. Show all posts
Showing posts with label Knime. Show all posts
Thursday, September 20, 2012
Remove duplicated rows in Knime
Goal
Delete the duplicated rows in table.
Strategy
1. Use GroupBy node, refer to
Detect and delete duplicate files (rows) based on two variable identifiers (date and lot number) and not based on row ID
2. Use Database Query. SELECT DISTINCT
3. Use code snippet in scripts: R, JPyhton, and Java to deal the table
Comment
1. Delete empty rows is a special case of this topic.
2. Two rows are duplicated at all the columns, or only on several columns (combination key)
3. S2, and S3 are not tested
Tuesday, January 31, 2012
Embedding data in workflow
Source: QnA in Knime forum
Goal
To embed data in workflow, so it can be distribute with the workflow package zip file.
Contributor ID: jfalgout
Goal
To embed data in workflow, so it can be distribute with the workflow package zip file.
Strategy
Create a directory with files underneath the directory containing the
node artifacts for a workflow, the "knime.node" flow variable will be
populated the next time you edit the node.
The folder you create within the node's folder has to be named "drop". Otherwise when you save the workspace whatever folders/files you add get deleted.
Test
Not yet
Sunday, January 15, 2012
Example to create new Node
Example to create new Node: http://tech.knime.org/developer/example
Monday, January 9, 2012
Typical workflow for test on specific Culumn
Source: QnA in Knime forum
Contributor ID: James Davidson
Derive a new column from a test on existing column
Strategy
1. Use [Row Filter] to do the test and split (with Include & exclude) the original table
2. Use [Java snippet] to generate the value for the new column
3. Use [Concatenate] to merge table back
Comment
1. In some case, [Joiner] my be a shortcut for the goal
2. Do not known whether it is the best solution, since I am just a beginner
Resource
Download James's Package
Resource
Download James's Package
Friday, January 6, 2012
Remove empty columns in Knime
Goal
Delete the columns comprise of blank value (0, NaN, NA, NULL or fix string)
Strategy
1. If the target is number column (double or int), use [Low Variance Filter] node
2. Use [Transpose] + [Row Filter] + [Transpose] , suggested by
http://tech.knime.org/forum/knime-general/removing-columns-where-every-value-is-empty
3. Use code snippet in scripts: R, JPyhton, and Java to deal the table
Comment
None of the three works for me.
1. My target columns are string type. [String to Number] node need to be wired to source column manually.
It does not make sense if I have tons of empty columns.
2. Missing Value in [Row filter]'s setting only works on specific column, which implies it is not an automation.
3. I can write R, Python or script outside Knime to do filtering.
Final solution
1. Pre-process the CSV file outside Knime
2. Manually skip the column in [File Reader] setting.
Delete the columns comprise of blank value (0, NaN, NA, NULL or fix string)
Strategy
1. If the target is number column (double or int), use [Low Variance Filter] node
2. Use [Transpose] + [Row Filter] + [Transpose] , suggested by
http://tech.knime.org/forum/knime-general/removing-columns-where-every-value-is-empty
3. Use code snippet in scripts: R, JPyhton, and Java to deal the table
Comment
None of the three works for me.
1. My target columns are string type. [String to Number] node need to be wired to source column manually.
It does not make sense if I have tons of empty columns.
2. Missing Value in [Row filter]'s setting only works on specific column, which implies it is not an automation.
3. I can write R, Python or script outside Knime to do filtering.
Final solution
1. Pre-process the CSV file outside Knime
2. Manually skip the column in [File Reader] setting.
Subscribe to:
Posts (Atom)
