Excel Knime 072018
Excel Knime 072018
Excel Knime 072018
All rights reserved. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any
form or by any means, electronic, mechanical, photocopying, recording or likewise.
KNIME Press
Technoparkstr. 1
8005 Zurich
Switzerland
knimepress@knime.com
3
Table of Contents
General Usage ....................................................................................................................................................................................................................................................... 6
Spreadsheets Workflows and Nodes ............................................................................................................................................................................................ 7
Folders Workspace .......................................................................................................................................................................................................................... 8
The KNIME Workbench ..................................................................................................................................................................................................................................... 9
Display Data Table ........................................................................................................................................................................................................................................... 10
Input/Output ....................................................................................................................................................................................................................................................... 11
Opening an Excel File Excel Reader Node ........................................................................................................................................................................................ 12
Opening a CSV File File Reader Node ............................................................................................................................................................................................... 13
Importing Content from Multiple Files of the same Type to a Single Table ................................................................................................................................................... 14
Importing Content from Multiple Sheets into a Single Table.......................................................................................................................................................................... 15
Connect to a Database Database Connector Nodes ........................................................................................................................................................................ 16
Tips on Reading Data with KNIME ................................................................................................................................................................................................................... 17
Appending / Joining Data .................................................................................................................................................................................................................................... 18
Appending Data Concatenate Node ............................................................................................................................................................................................... 19
VLOOKUP Filter and Joiner Node .................................................................................................................................................................................................. 20
Filtering and Transformations ............................................................................................................................................................................................................................. 21
Filtering / Removing Rows with a Specific Value Row Filter................................................................................................................................................................. 22
Filtering / Removing Rows with Different Values Rule-based Row Filter ............................................................................................................................................. 23
Sorting Rows by Multiple Key Columns Sorter Node .......................................................................................................................................................................... 24
Removing Columns Column Filter Node .......................................................................................................................................................................................... 25
Reordering and Renaming Columns Column Resorter and Column Rename Node ........................................................................................................................... 26
Changing Data Types String to Number, Number to String, Strings to Date&Time ......................................................................................................................... 27
Aggregations .................................................................................................................................................................................................................................................... 28
Pivot Tables Pivoting Node ........................................................................................................................................................................................................... 29
Pivot Table without Columns GroupBy Node ................................................................................................................................................................................... 30
4
Math Functions and Text Functions ................................................................................................................................................................................................................ 31
Math Functions Math Formula Node .............................................................................................................................................................................................. 32
Math Functions Math Formula Node ............................................................................................................................................................................................. 33
Concatenation and Find& Replace String Manipulation Node ............................................................................................................................................................ 34
5
General Usage
6
Spreadsheets Workflows and Nodes
Excel KNIME Analytics Platform
Microsoft Excel is a spreadsheet program, which features calculation, graphing KNIME Analytics Platform implements visual programming. This means that each
tools, pivot tables, and a macro programming language (Visual Basic for data analysis step is represented by means of an icon block, called a node, in a
Applications, VBA for short). graphical editor.
By using cell mathematics, macros, and VBAs you can edit a sheet. This can be A sequence of connected nodes is called a workflow and is the corresponding
really easy cell mathematics, like summarizing the values from cells A1 and B1 (= concept of an Excel sheet with many functions and/or VBAs.
SUM(A1, B1)), but can be also really complex, embedded logic.
Data is organized through data tables, where each data cell is identified by a
column header and a Row ID. To visualize the content of a data table, see page 10.
Note. Nodes have four possible states displayed by a little traffic light under the node itself:
- Not configured -> red light
- Configured -> yellow light
- Successfully executed -> green light
- Executed with error -> red light with cross
7
Folders Workspace
Excel KNIME Analytics Platform
Excel files are normally saved in different folders. A single Excel file can contain The workspace defines the folder where all workflows, data and intermediate
data are saved. One workflow corresponds to an Excel sheet with all formulas,
multiple sheets. visualizations and VBAs. All the projects and datasets saved in a workspace are
available in the KNIME Explorer, located in the top left corner of the KNIME
workbench. The path to the workspace is selected at the very beginning, after
starting KNIME Analytics Platform.
You can still change the workspace after KNIME has been launched, by going to
“File” in the top menu and selecting “Switch Workspace”. You can have multiple
workspaces, e.g. for different projects or customers.
8
The KNIME Workbench
Start KNIME Analytics Platform from the desktop or from the installation folder. The KNIME workbench opens, which includes: “Workflow Editor”, “KNIME Explorer”,
“Workflow Coach”, “Node Repository”, ”Node Description”, “Outline”, and “Console”.
“KNIME Explorer” shows the list of currently available workflows and KNIME servers for the selected workspace.
“Workflow Coach” recommends the next node based on the KNIME user statistics.
“Node Repository” panel contains all available nodes. A “Search” box is available at top of this panel to search for nodes.
“Workflow Editor” in the center allows for the creation and editing of workflows.
“Node Description” shows text describing the node task and configuration settings, for the selected node either in the “Workflow Editor” or in the “Node Repository”
panel.
The “Outline” panel offers an overview of the workflow and the “Console” panel shows execution messages.
KNIME workflows are created by dragging&dropping of nodes from the “Node Repository” or “Workflow Coach” panel into the “Workflow Editor”.
Nodes are connected to each other through their input and output ports. Just click the output port of the first node and release at the input port of the second node.
Nodes that just have been created show a red light status: not yet configured. To configure a node, right-click the node and select the option “Configure” or alternatively
double-click the node. The node “Configuration” window opens. Configure the node. If the configuration is successful, the node status changes to a yellow traffic light.
The node is now configured, but not yet executed. To execute the node, right-click the node and select the “Execute” option. If the execution is successful, the node
changes its status to a green light.
Node Description
Workflow Coach
Outline
9
Display Data Table
Excel KNIME Analytics Platform
In Excel what you see is what you get. This means that the data table you see is The output data tables produced after node execution are always available. To
the final data table. see them:
- Right-click the node in the workflow
- Select the last option in the context menu
Note. Some nodes, like plotting and modeling nodes, also have a more complex “View”
function. The option leading to this “View” is usually displayed in the middle of the context
menu.
10
Input/Output
11
Opening an Excel File Excel Reader Node
Excel KNIME Analytics Platform
To open an Excel file you can either double click on the file or open Excel, go to The “Excel Reader (XLS)” node reads a single sheet of an Excel
“File” in the top menu, select “Open…” and then browse to the file you want to file.
open. As in Excel you first have to define the file path. Next you can
select the sheet. The additional setting options enable you to
define whether the table has column headers and / or row IDs
and to specify which part of the sheet you want to read.
Use the Refresh button to update the data preview.
File
Path
Sheet
Name
Column Headers
Note 1: Possible reading formats are: Integer, Double, and String. A date must be read as
String and later on converted to Date&Time type with a “String To Date&Time” node.
Note 2: With a loop you can automatically read all sheets of an Excel file.
12
Opening a CSV File File Reader Node
Excel KNIME Analytics Platform
To open a CSV file click on “File” in the top menu and select “Import”. Select CSV The “File Reader” node reads various text based files, e.g. CSV
file and then choose which file you want to open. files. In the configuration window you can set all the necessary
Fill in all necessary options, such as the separating character, to correctly read options, such as separating character, to correctly read the
the file. file.
File
Path
Delimiter
Column
Headers
Column Type
Note 1. There are many possible reading formats. Basic formats are: Integer, Double, and
String. A date must be read as String and later on converted to Date&Time type with a
“String To Date&Time” node.
Note 2. KNIME has also a specific “CSV Reader” node dedicated to reading CSV files.
13
Importing Content from Multiple Files of the same Type to a Single Table
Excel KNIME Analytics Platform
If you have a folder containing multiple files of the same type (e.g. CSV), you can With a simple loop you can automatically read multiple files of the same type.
open them all at the same time in different Excel instances. Follow the
instruction from the previous page and select all the files you want to open. The “List Files“ node creates a list of the documents in a folder and their URL. In
addition to the directory, you can specify the extension, so that only documents
To move data tables together into one single sheet, you have to proceed of this type are listed.
manually using copy and paste. The loop (the part in between the blue nodes) reads one document at each
iteration. Therefore the “Table Row to Variable Loop Start” node creates a flow
Note: Before copying and pasting, ensure that all files have the same column order. variable with the URL of the next document at each iteration. This flow variable
is used in the “Table Reader” node to control the “Input location”. The “Loop
End” node concatenates the content of the different documents into one single
output table.
Note 1: Section 7.1 of the free E-Learning Course gives an introduction to flow variables.
Note 2: Section 7.2 of the free E-Learning Course gives an introduction to loops in
KNIME.
14
Importing Content from Multiple Sheets into a Single Table
Excel KNIME Analytics Platform
To move data from multiple Excel sheets into one Excel sheet you proceed With a simple loop you can read all sheets of an Excel file automatically.
manually, using copy and paste.
The “Read Excel Sheet Names (XLS)” node creates a list of all sheet names. The
Note: Before copying and pasting, ensure that all files have the same column order. loop (the part in between the blue nodes) reads one sheet of the Excel file at
each iteration. Therefore, at each iteration the “Table Row to Variable Loop
Start“ node creates a flow variable with the sheet name as its value. This flow
variable is used in the “Excel Reader (XLS)” node to control the sheet selection.
The “Loop End” concatenates the content from the different tables.
Note 1: Section 7.1 of the free E-Learning Course gives an introduction to flow variables.
Note 2: Section 7.2 of the free E-Learning Course gives an introduction to loops in
KNIME.
15
Connect to a Database Database Connector Nodes
Excel KNIME Analytics Platform
Note 1: The approach to database reading is modular. The three phases (connect, SELECT,
extract) can be run altogether in a “Database Reader” node or built step by step with a
“Database Connector”, a “Database Table Selector”, and a “Database Connection Table
Reader” node. And there are more database nodes to help build the SQL query.
Note 2: Database nodes only builds the SQL query, they do not execute it. Only the final
node extracting the data executes the SQL query.
Note 3: A number of database connector nodes are available to connect to the most
commonly used databases. However, the “Generic Database Connector” node allows you
to connect to any database via JDBC driver.
16
Tips on Reading Data with KNIME
All reader nodes require paths to the input file location. Let’s collect some Tips&Tricks for this:
Data files saved in the workspace folder are available in the KNIME Explorer panel (top left). To read in one of these files, you can just drag&drop a file from the KNIME
Explorer panel to the workflow editor. KNIME automatically creates the correct reader node and sets the path of the input location.
In KNIME we have different options to provide a file path. This becomes important when you start sharing your workflows or exporting them to other KNIME Analytics
Platform installations or KNIME Servers. Besides the absolute URL, the knime:// protocol accesses locations relative to the current workflow folder, workspace folder,
and server mountpoint.
Absolute URL
Mountpoint-relative URL
Local Path
17
Appending / Joining Data
18
Appending Data Concatenate Node
Excel KNIME Analytics Platform
Select manually the area you want to append and copy and paste the content
below the first table.
Note 1: Before copying and pasting ensure that all tables have the same column
structure.
19
VLOOKUP Filter and Joiner Node
Excel KNIME Analytics Platform
The VLOOKUP function is used for different tasks. The most common uses are: 1. Look up a certain value, e.g. the price of a certain product.
2. Join columns based on a primary key (look up value), e.g. join product
information based on the product ID.
Note 1: Your full original table is still available at the output port of the “Table Reader”
An alternative function for the second task is INDEX MATCH. node. See more information about the “Row Filter” and “Column Filter” node on pages
22 and 25.
2. Join columns based on a joining column, e.g. join product information based
on product ID.
Filter Columns
Joining Mode
Note 1: Similarly to databases different joiner modes are available: Inner, left outer, right
outer and full outer join. The join mode that corresponds to the VLOOKUP is the left
outer join (missing values in case there is no match in the range containing the look up
value).
Note 2: The “Joining Column” is the look up value column.
20
Filtering and Transformations
21
Filtering / Removing Rows with a Specific Value Row Filter
Excel KNIME Analytics Platform
To filter data select a random cell in the data table, go to the Home tab, click the In KNIME there is no difference between filtering and
“Sort& Filter” button and select “Filter”. removing rows, as the original table is not deleted and is still
available at the output port of the previous node.
Filter Column
Select the value you are interested in from the drop down menu. Value to in- or exclude
Note 1: If you only interested in the rows with one specific value you can use the “Row
To remove rows, select the rows you want to delete, right click and select Filter” node.
“Delete Rows”. Note 2: If you want to include rows based on different values you can use the “Rule-
based Row Filter”. (See next page).
Note 3: On the right you can choose whether you want to include or exclude the rows
with the matching value
Note 4: Further filter options are available, e.g. on a numerical range, filter rows by row
number or row ID, or missing values only.
22
Filtering / Removing Rows with Different Values Rule-based Row Filter
Excel KNIME Analytics Platform
To filter data select a random cell in the data tables, click on “Sort& Filter” and In KNIME there is no difference between filtering and
select “Filter”. removing rows, as the original table is not deleted and is still
available at the output port of the previous node.
List of Columns
Select the values you are interested in from the drop down menu.
List of Functions
Expression
To remove rows you have to select the rows you want to delete, right click and Note 1: At the bottom of the configuration window you can choose whether you want to
choose “Delete Rows”. include or exclude TRUE matches.
Note 2: The “Rule-based Row Filter” node has a number of different functions for many
advanced filter options.
Note 3: Different rows in the expression frame function like an OR conjunction.
23
Sorting Rows by Multiple Key Columns Sorter Node
Excel KNIME Analytics Platform
To sort rows by multiple columns, select the columns by which you want to sort. With the “Sorter” node you can sort by one or more columns
Then click on Sort&Filter and select “Custom Sort”. in either ascending or descending order.
Note 1: You can add as many key-columns as you want by clicking the “new columns”
button.
24
Removing Columns Column Filter Node
Excel KNIME Analytics Platform
To remove columns just select and delete the superfluous columns. With the “Column Filter” node you can delete superfluous
columns. Remember that the full data table will be still
Remember: If you remove a column used in a calculation field this will break available at the output port of the previous node and for
your formula. calculations you run upstream.
Note 1: You can use the arrow buttons in the middle to move columns from the Include
to the Exclude frame and vice versa.
25
Reordering and Renaming Columns Column Resorter and Column Rename Node
Excel KNIME Analytics Platform
To reorder columns: Define the desired order of the columns by selecting the one you want to move
• Select the column header of the column you want to move and using the “Actions” buttons on the right.
• Press ctrl + shift + down arrow
• Click and hold the green outline of the column you want to move
• Drag your column to the desired position
To rename a column just click on the column cell and change the cell value. Double click the column you want to rename, activate the checkbox “Change”
and define the column header in the textbox.
Note 1: In addition to the column header you can also change the data type.
26
Changing Data Types String to Number, Number to String, Strings to Date&Time
Excel KNIME Analytics Platform
To change the data type of a column or a cell, select it and go to the home tab.
Then use the drop down menu in the middle.
Note 1: In the “String to Number” node you can choose between different numerical
types, e.g Double, Integer and Long.
27
Aggregations
28
Pivot Tables Pivoting Node
Excel
To create a pivot table in Excel The “Pivoting” node is configured via three tabs:
• Click on any single cell inside the data set • “Groups” defines the group columns, aka columns in
• Go to the Insert tab and click Pivot Table the “Rows”-frame in Excel (final row IDs)
• “Pivots” defines the pivoting columns, aka columns in
the “Columns”-frame in Excel (final column headers)
• “Manual Aggregation” defines the aggregation variables and the
aggregation methods
• Select table / range and output location The “Value” setting option corresponds to the “Manual Aggregation” tab.
The “Pivoting” node produces three output tables: the pivot table and the total
values for columns and rows.
Columns Values
Rows
• Choose pivot table fields by dragging them into “Columns”, “Rows” and
“Values”.
Note 1: The “Pivoting” node doesn’t have “Filter” options, but you can simply use a “Row
Filter” node beforehand.
Note 2: In KNIME you have to choose at least one column for the Groups and Pivots. In
case you want to choose only “Rows” you can use the “GroupBy” node.
29
Pivot Table without Columns GroupBy Node
Excel KNIME Analytics Platform
Create a pivot table as described on the previous page and drag only columns The “GroupBy” node is configured via two tabs:
into the “Rows” and “Values” frames. • “Groups” defines the group columns, aka columns in
the “Rows” frame in Excel (final row IDs)
• “Manual Aggregation” defines the aggregation
variables and the aggregation methods
The “Value” setting option corresponds to the “Manual Aggregation” tab.
Rows Columns
30
Math Functions and Text Functions
31
Math Functions Math Formula Node
Excel KNIME Analytics Platform
Here is a list of frequently used math formulas in Excel. You will find a translation The “Math Formula” node has a long list of functions. The
into the KNIME formulas on the next page: table on the next page provides a translation of the most used
• SUM Excel functions.
• AVERAGE
• MEDIAN Columns
• SUMPRODUCT
• ABS
• SUMIF
• Round functions
o ROUND
Funtions
o ROUNDUP
o ROUNDDOWN
Note 1: You can decide whether you want to append a new column or replace one of the
columns, by using the checkboxes underneath the Expression frame.
Note 2: By activating the checkbox “Convert to Int” you can ensure that your output
appended / replaced column is of type Integer.
32
Math Functions Math Formula Node
Excel KNIME Analytics Platform
ABS(number1) abs(Col) = The absolute value for all values in the selected column
Round functions:
ROUND(number1, num_digits) round(Col, NumberOf Digits) = Number of digits is optional.
ROUNDUP(number1, num_digits) ceil(Col*10^(num_digits))/10^num_digits
ROUNDDOWN(number1, num_digits) floor(Col*10^(num_digits))/10^num_digits
33
Concatenation and Find& Replace String Manipulation Node
Excel KNIME Analytics Platform
1. The function CONCATENCATE( text1, text2,..) combines different texts into With “String Manipulation” node you can manipulate columns
one cell. of type String. Therefore a lot of different functions are
available.
2. To find and replace a certain value you have to go to “Edit -> Find ->
Replace…” to open the dialog below.
In the dialog that opens, you can define the value you want to replace and the
value you want to replace it with.
Note 1: You can either join the values of different columns row wise, or add also another
String value to a column.
2. Find&Replace
Note 1: String is the String value where you want replace something. This can be a
column to apply the replace row wise or a fixed String value.
34
35
The KNIME Booklet for Excel Users
Are you an experienced Excel user and want to start using KNIME Analytics Platform?
It’s sometimes difficult to switch from one software tool to another. But this booklet is
the perfect starting point as it maps the most commonly used Excel functions and
techniques to their KNIME equivalents. Find out, for example, how data reading, filtering,
sorting, and vlookup work in KNIME.
For a complete introduction to KNIME, please refer to my book “KNIME Beginner’s Luck”
available from KNIME Press under https://www.knime.com/knimepress
Kathrin Melcher is currently a Data Scientist at KNIME. She holds an MSc in Mathematics,
from the University of Konstanz, Germany. She joined the KNIME Evangelism team in May
2017 and has a strong interest in data science, machine learning, and algorithms. She
enjoys teaching and sharing her knowledge on these topics.
36