Graphical User Interface

GUI

The main interface of the tool is designed to be simple and functional, yet powerful at the same time. The user can select which modules to enable by selecting the corresponding checkbox. A menu with different options is available to perform different actions. The interface was created using PyQt5.

When the app opens the user needs to select the options that should be enabled and press the “Start Logger” button. Then all selected modules are started. The configuration of each module takes place in the Main file.

Note

The primary methods of this class to work with are:

• onButtonClick() - triggered by ‘start logger’ button

• handleProcessMining() - intermediate process event log

• choices() - perform all the analysis on the event log,

class modules.GUI.GUI.MainApplication(parent=None)

GUI

PMThreadComplete(result)

Once the working thread started by handleMerge() finishes processing, the result is returned in this method, which in turns calls the choices() method

__init__(parent=None)

Initialize GUI

cancelProgressDialog()

manually dismiss progress dialog

choices(pm, log_filepath)

This method is called once the worker thread started by handleMerge() completes.

It is the core of the tool, responsible for performing all the analysis on the event log, from log processing to RPA SW robot generation.

If process discovery is enabled in preferences:

• high level flowchart is generated from process mining class

• high level petri net is generated from process mining class

• events are logged in status queue

If decision points analysis is enabled and there are at least 2 traces in the event log

• high level DFG is opened

• decision points class is called, the resulting decision dataframe is stored

• new flowchart is generated from decided dataframe

• choices dialog class is called, taking as input decided dataframe

• final flowchart with user edits is generated

• python SW robot is generated from RPAScript path

• Uipath project is generated from UIPathXAML class

Parameters:

• pm – instance of the Process Mining class

• log_filepath – path of the event log to analyze

compatibilityCheckMessage()

Display message if a program like office, firefox, chrome, edge, opera is not installed in the operating system.

createBottomLayout()

create bottom layout that contains button to start logging

createBrowserLoggerGroupBox()

Create checkboxes to log browser events in chrome, firefox, edge or opera

createMenu()

Create file menu with preferences option, supervision option and options to merge multiple CSV event logs and to analyze one event log.

createOfficeLoggerGroupBox()

Create checkboxes to log microsoft office applications like excel, word, powerpoint

createProgressDialog(title, message, timeout=None)

Create progress dialog displayed when starting or stopping the action logger

Parameters:

• title – title of progress dialog

• message – message of progress dialog

• timeout – timeout after which progress dialog automatically disappears

Returns:

progress dialog

createStartButton()

create button to start and stop action logger

createStatusLayout()

create status layout, the window below the start button where all logging appears

createSystemLoggerGroupBox()

Create checkboxes to log operating systems events like opening a program, editing files/folders, logging clipboard, hotkeys or usb drives insertion

createTopLayout()

create top layout with buttons to enable all modules at once

excelDialog()

Displays file dialog when excel is selected in order to choose a file to open

handleCheckBox()

Detect which modules should be run based on selected checkboxes in UI

handleDataQualityCheck()

This method is called when the user wants to check the data quality of a UI log and get static noise filters applied on the log.

1. It displays a file dialog to select the file. The method returns multiple files that can be checked for data compliance.

2. After the event log(s) have been selected, the method starts a worker thread to process them using the staticNoiseIdentification in the utils.py. This can run in the background and after compilation display a new PyQT window.

3. Once the processing is complete a new window with errors is displayed.

:param

handleMerge(merged=True, title='Select multiple CSV to merge', multipleItems=True)

This method is called when the user wants to merge multiple csv files or run an event log (in this case it is called by the handleRunLogAction() method).

1. It displays a file dialog to select the event logs to process; this dialog returns a list of paths, each one representing the location of an event log.

2. After the event logs have been selected, the method starts a worker thread to process them using the handleProcessMining() method. A new thread is necessary in order not to block the main UI.

3. Once the thread completes, the choices() method is called.

Parameters:

• merged – boolean value indicating if the user wants to merge multiple files or to run a single event log

• title – title of the file dialog

• multipleItems – boolean value indicating the ability to select multiple items

handlePreferences()

Calling this mentod shows the preferences dialog window

handleProcessMining(log_filepath: list, merged=False, fromRunCount=False)

This method can either be called by handleMerge() (when the user wants to merge multiple CSV or run RPA from log) or by the GUI after the user presses the Stop logger button.

1. It creates an instance of the Process Mining class, passing the selected event log files.

2. Then it returns the instance of the class along with the selected file path.

Parameters:

• log_filepath – list of paths pointing to event logs selected in the file dialog

• merged – boolean indicating if the logs have been merged

• fromRunCount – indicates whether the method has been called by handleMerge() or by the GUI

Returns:

process mining class instance, log_filepath

handleRPA(log_filepath)

Generates an RPA Script from the log file under log_filepath

Parameters:

log_filepath – Filepath to a csv file containing a UI log

handleRunCount(log_filepath)

Merges multiple traces of execution together generating one event log. It depends on “number of runs after which event log is generated” setting in preferences.

Parameters:

log_filepath – path of event log file

handleRunLogAction()

This method is called when the user wants to perform RPA analysis on an event log.

It calls the handleMerge method with parameters (merged=False, title=’Select CSV to run’, multipleItems=False)

onButtonClick()

Main method executed by the tool to start log capture when start button is clicked by user.

If tool is starting:

• set boolean variable ‘running’ to true

• check which checkboxes are enabled in the GUI

• if excel checkbox is selected, display dialog to select file to open (deprecated)

• start progress dialog indicating loading

• start main process, passing as parameters the boolean checkbox values

If tool is stopping:

• set boolean variable ‘running’ to false

• kill active processes using their PID before closing main

• terminate main

• call handleRunCount() method to check counter and generate event log file

platformCheck()

set appropriate values based on platform.

disable checkboxes not available on macOS like word and powerpoint.

setAppIcon()

set app icon with support to dark mode

setCheckboxChecked()

triggered by “enable all” button on top of the UI in some cases the checkbox should be enabled only if the program is installed in the system

setStyle()

set native GUI style for each OS

showAboutMessage()

Show information about the tool under File menu

showHints()

Show best practice hints for using smartRPA curated by the developers

staticNoiseFilter(uiProcessingFlag: bool) → DataFrame

Gets a UI log and checks for attribute noise using standard format definitions. The cleaned dataframe does not contain attribute values that are incorrectly formated

Parameters:

• uilog – User interaction log dataframe

• uiProcessingFlag – True if the result should be displayed in a UI, false if only the log should be

Returns:

Cleaned dataframe

updateListWidget()

Reads status queue and updates log window below start logger button in the GUI

updateStartButtonState()

Controls the status of the “start logger” button. If any CB is clicked it does update the status. If any CB is active, the button stays active.

Preferences window

User preferences are stored in a local preferences file, because they need to remain consistent across different runs. The cross-platform Python package nativeconfig2 was used. It uses native mechanisms such as Windows Registry or NSUserDefaults to store user settings.

The first option allows the user to enable or disable Process discovery analysis described in Section 4.6 after the logs has been recorded. This is useful when the tool is used in a distributed environments where multiple people record defined steps that will later be merged together and analysed, as described in Chapter 5.

The second option allows the user to set the number of runs after which the logs are merged XES file is generated. This option is useful when the user wants to perform multiple runs at the same time to analyze different variations. After reaching the specified number of runs, the tool merges all recorded CSV into one log file and converts it to XES, as described in Subsection 4.4.

class modules.GUI.PreferencesWindow.Preferences(parent, status_queue)

Preferences window

__init__(parent, status_queue)

Initialize preferences window.

It allows to:

• enable or disable process discovery analysis

• select the number of runs after which event log is generated

• select analysis type (either decision points or most frequent routine)

• Future: enable or disable the screenshot recording feature

Parameters:

status_queue – queue to send messages to main GUI

handleButton()

Closes preference window

handle_cb()

Sets process discovery option in config.py Triggered after number of GUI Runs specified in preferences

handle_cb_scrsht()

Sets screenshot option in config.py If enabled each event stored creates a screenshot

handle_cb_supervision()

Sets supervision option in confi.py If enabled, after each event the user is asked for tagging the event

handle_radio()

Sets most frequent or decision point analysis in config.py

handle_slider()

Sets number of runs before mining in config.py

class utils.config.MyConfig

Configuration class, used to store preferences.

Preferences include:

• total number of runs before executing process mining analysis

• control process discovery, if disabled csv is generated but process discovery techniques are not applied

• enable most frequent routine analysis (deprecated)

• enable decision points analysis

File dialog

A native file dialog to select files has been developed using the QFileDialog class available in PyQt5. It is used to select event logs to merge or to run. It supports both single and multiple selection, and it is possible to specify the file types that can be selected, in this case only CSV log files. It returns a list of strings representing the path of each file selected.

modules.GUI.filenameDialog.getFilenameDialog(customDialog=True, title='Open', hiddenItems=False, multipleItems=False, isFolder=False, forOpen=True, directory='', filter_format='')

Build custom dialog to select multiple files

Parameters:

• customDialog – if False, use native system dialog, otherwise use custom dialog

• title – dialog title

• hiddenItems – display hidden items

• multipleItems – allow selection of multiple items

• isFolder – allow selection of folders

• forOpen – select files for opening or saving

• directory – start directory

• filter_format – filter to select only certain file types (for example *.csv)

Returns:

path of selected file (list of paths if multiple files are selected)

Choices dialog

Once the routine to automatize is selected, the user can customize its editable fields through a custom dialog window. The program automatically recognises which fields are editable, such as typing something in a web page, renaming a file or pasting a text, and dynamically builds the GUI to let the user edit them. After confirmation, the dataframe is updated.

This custom dialog has been implemented extending the QDialog class from PyQt5. It receives as input the low-level most frequent routine dataframe previously computed. This dataframe is then filtered to select only the events that needs to be edited by the user.

Choices dialog allows the user to customize editable fields before generating the SW robot.

modules.GUI.choicesDialog.ChoicesDialog.__init__(self, df: DataFrame)

Parameters:

df – low level dataframe of decided trace or most frequent routine

modules.GUI.choicesDialog.ChoicesDialog.addRows(self)

Add editable rows to custom dialog, such as typing in a web page, renaming a file or pasting a text.

For each row there is a label, the description of the field, and a value, the element to be edited

modules.GUI.choicesDialog.ChoicesDialog.handleReturn(self)

Called when choices dialog is closed (OK button is pressed)

Each row of the dataframe in input is updated with the values inserted by the user.

Decision dialog

The user can choose which user actions enact by means of a decision dialog, which displays data from the decided dataframe, that is used to group together all the rows of a routine trace belonging to a routine fragment into a single line.

class modules.GUI.decisionDialogWebView.DecisionDialogWebView(df: DataFrame)

Decision dialog to display and choose different decision in a routine.

The input dataframe is shown in a HTML table generated at runtime using the dataframeToHTML() function and displayed in a webview.

__init__(df: DataFrame)

Initialize decision dialog

Parameters:

df – keywords dataframe

Returns:

caseID of selected trace

modules.GUI.decisionDialogWebView.dataframeToHTML(keywordsDataframe: DataFrame)

Convert input dataframe to HTML table

Table is generated using BeautifulSoup.

Parameters:

keywordsDataframe – keywrods dataframe

Returns:

HTML page with table

Supervision dialog

The user can annotate the actions he/she has done using the supervision feature enabled in the preferences window.

modules.supervision.getResponse(json_str='')

Takes a dict object containing the key-values of the recorded user action. Displays a UI to the user for feedback on the relevancy of the previous action.

Parameters:

json_str – String containing a dictonary with event key-values

Returns:

Response TRUE, FALSE, NONE

Return type:

bool