ERGONOMICS 2 (COGNITIVE ERGONOMICS)

Laboratory Activity No. 2

Name of the Activity: FRAMING THE STATEMENT OF THE PROBLEM

Student’s Last Name and First Name: Saligo Rogie, G_________________________________

Year and Section: 3-E Date of Submission:04/24/2024

I. TITLE OF THE USABILITY STUDY

Comparative Usability Testing Research of DeepAI and SciteAI: A User Interface (UI) and User
Confidence (UC) Perspective

II. MAIN PURPOSE OF THE USABILITY STUDY

The study aims to evaluate DeepAI and SciteAI's user interface (UI) and user confidence
(UC) from a user-centric perspective. It will assess the UI design, user experience, and
confidence levels of both platforms. The study will identify potential issues and assess the
effectiveness of UI elements. It will also measure user confidence by examining factors
influencing confidence, such as accuracy, reliability, and ease of use. The findings will enhance
the usability and user confidence of both platforms.

III. STATEMENT OF THE PROBLEM (Attach a separate sheet or add additional sheets as
needed.
Artificial intelligence (AI) and machine learning (ML) technologies are growing quickly.
DeepAI and SciteAI are two popular tools that use AI for tasks like understanding language and
analyzing data for scientific research. This research aims to conduct a comparative usability
testing of DeepAI and SciteAI, two popular AI tools in generating outputs evaluating its user
interface, accuracy and reliability.

The study seeks to answer the following questions:

Effectiveness:
1. To what extent do the outputs generated by DeepAI and SciteAI in scientific research
align with expected precision, correctness, and consistently reliable ensuring informed
decision-making and user trust?

Efficiency:
2. To what extent do users accomplish tasks more quickly and efficiently to generate others
using the DeepAI and SciteAI?
User satisfaction:
3. How does the use of DeepAI and SciteAI impact user experience and productivity,
specifically in terms of ease of navigation, clarity of design elements, and overall user
satisfaction?

By addressing the identified problems and providing valuable insights into the accuracy,
reliability, and user experience of DeepAI and SciteAI platforms. This analysis will help in
ensuring informed decision-making and enhancing user trust and satisfaction.
ERGONOMICS 2 (COGNITIVE ERGONOMICS)

Laboratory Activity No. 3

Name of the Activity: DEFINING EFFECTIVENESS AND DESIGNING DATA
COLLECTION TOOL/S

Student’s Last Name and First Name: Saligo Rogie___________________________________

Year and Section: 3-E DateSubmission:04/24/2024

I. TITLE OF THE USABILITY STUDY

Comparative Usability Testing Research of DeepAI and SciteAI: A User Interface (UI) and
User Confidence (UC) Perspective______________________________________________

I. STATEMENT OF THE PROBLEM component relating to EFFECTIVENESS

The purpose of the usability study's effectiveness component is to evaluate how well
users generate output in DeepAi and SciteAi. It focuses on how accurate and reliable the
outputs are being generated.

The study seeks to answer the following questions:

1. What level of correctness do the data analysis insights provided by DeepAI and SciteAI
exhibit, and how accurately do they reflect the underlying patterns and trends in the data?
2. How reliably do the research findings generated by DeepAI and SciteAI align with
established methodologies, citations, and research articles, ensuring credibility and
trustworthiness?
3. To what extent do users perceive the outputs generated by DeepAI and SciteAI as reliable
and trustworthy, and how does this perception influence their willingness to rely on these
outputs for decision-making purposes?

II. Definition of EFFECTIVENESS in the Context of the Usability Study

In the field of usability studies, effectiveness is the fundamental measure of how well
users can complete their intended tasks using a product or system. It delves into the practical
aspects of user interaction, focusing on whether users can achieve their objectives within the
given context and constraints. In the context of Comparative Usability Testing Research of
DeepAI and SciteAI: A User Interface (UI) and User Confidence (UC) Perspective is defined
and evaluated in relation to how well these AI systems perform in meeting user needs, achieving
usability goals, and enhancing user confidence in reliability and accurateness of generated
output. Effectiveness in this context can encompass several key aspects:
 Accuracy of Output
This evaluates how well the outputs generated by the system, including language
translations, data analysis insights, and research findings, meet predetermined standards of
precision and correctness. High accuracy ensures that users can make informed decisions
based on the output, fostering trust in the system's capabilities.

Reliability of Output
This measures the consistency and dependability of the system's outputs across different runs
and under varying conditions. Factors considered include the stability of underlying
algorithms, the system's ability to handle diverse inputs and scenarios, and the
reproducibility of results over time. A reliable system instills confidence in users by
consistently delivering expected outcomes without unexpected errors or variations.

III. DATA COLLECTION PROCEDURE concerning EFFECTIVENESS parameter

(Add additional sheets as needed.)

1. Comparative Analysis: In the context of Comparative Usability Testing Research of

DeepAI and SciteAI: A User Interface (UI) and User Confidence (UC) Perspective
comparing the outputs generated will validated data or research to assess their
correctness, accuracy, and alignment with established patterns and trends. This can be
done through statistical measures like error rates.
2. Citation and Reference Analysis: For research findings, the user will be ask to perform
a task in generating citations and evaluate the citations and references provided by
DeepAI and SciteAI can indicate their credibility and trustworthiness. This involves
verifying the sources cited by the AI tools and assessing their relevance and reliability.

IV. DATA COLLECTION INSTRUMENT/S or TOOL/S in relation to the

EFFECTIVENESS parameter of the specific usability study (Attach Questionnaire,
Observation Sheet, Checklist, Data Sheet, etc.)

Checklist:

Welcome to Usability Testing Research in titled Comparative Usability Testing

Research of DeepAI and SciteAI: A User Interface (UI) and User Confidence (UC)
Perspective. This usability checklist is tailored to assess the effectivenss of DeepAI and
SciteAI. This research delves into the User Interface (UI) and User Confidence (UC) aspects of
both platforms, aiming to provide comprehensive insights into their usability. Below are the
key points included for assessment.
 USER GROUP 1 USER GROUP 2

Trait no. 1 Not technologically Literate Technologically Literate

Trait no. 2 No Prior Exposure on Ai’s Accustomed on ai platforms

Trait no. 3 No prior experience with Accustomed on using deepai and sciteai

Trait no. 4 Age range(16-20) Age range(20-29)

Trait no. 5 High School Student College Student

Name:

ID no.:

Gender:

User Group:

No. Questions Yes/No

1
Are you confident with the accuracy of the
information provided by DeepAI/SciteAI for
this task?

2
Were there instances where the system’s
output differed significantly when inputs
were slightly adjusted?
3
Did the outputs from DeepAI/SciteAI seem
complete and address all relevant aspects of
the task?

4
Are there challenges or issues have you
encountered in opening the system

5
Have you noticed any instances where the
information provided by Deepai/Sciteai did
not meet your expectation in terms of
accuracy?

6
Did you encounter any information in the
DeepAI/SciteAI outputs that you questioned
or doubted its correctness?

EFFECTIVENESS DATA SHEETS:

TASK I: Open the DeepAi and SciteAi

No. Subtask Success (YES/NO)

1 ● Open electronic devices and connect to internet

2 ● Open browser or chrome

3 ● Click the search box and type scite.ai then enter

4 ● Click the DeepAi or SciteAi website

5 ● Click the assistant icon

6 ● Sign in with your personal email

7 ● Click ok
 8 ● Click the assistant icon

9 ● Scroll down then click the search box

TASK 2: Generate output in DeepAi and Sciteai and evaluate the accuracy and reliability

No. Subtask Success (YES/NO)

1 ● Click the search bar

2 ● Type in the search bar what you want to search

3 ● Click the arrow icon

4 ● Evaluate the accuracy and reliability of outputs

5 ● Compare the generated data to existing data in

scientific research to measure accurateness

6 ● Input questions looping where users generate the

outputs with the same questions to identify any
inconsistency or issues with the reliability of the
outputs
ERGONOMICS 2 (COGNITIVE ERGONOMICS)

Laboratory Activity No. 4

Name of the Activity: DEFINING EFFICIENCY AND DESIGNING DATA COLLECTION
TOOL/S

Student’s Last Name and First Name: Saligo Rogie

Year and Section: 3-E Date of Submission:04/24/2024

I. TITLE OF THE USABILITY STUDY

Comparative Usability Testing Research of DeepAI and SciteAI: A User Interface (UI) and User
Confidence (UC) Perspective_____________________________________________________

II.STATEMENT OF THE PROBLEM component relating to EFFICIENCY

The user interface (UI) design of AI platforms such as DeepAI and SciteAI plays a
crucial role in shaping user experience and perception. This study aims to investigate the
differences in UI design between DeepAI and SciteAI, focusing on navigation, visual appeal,
ease of use, and their impact on user confidence and perception of AI-generated outputs.

1. How does the UI design of DeepAI and SciteAI differ in terms of navigation, visual
appeal, and ease of use?
2. What factors contribute to user confidence when interacting with AI outputs from
DeepAI and SciteAI?
3. How do variations in UI elements impact user perception of AI-generated insights
and recommendations?

Understanding the nuances of UI design and its influence on user confidence and perception is
vital for optimizing AI platforms like DeepAI and SciteAI. By addressing these research
questions, this study seeks to provide actionable insights into UI improvements that can enhance
user experience, trust, and engagement with AI-generated outputs.

III. Definition of EFFICIENCY in the Context of the Usability Study

Efficiency refers to how well users can complete tasks with a product or system while
making the best use of available resources, such as time, effort, and cognitive load. It focuses on
the relationship between the outcome (task completion) and the resources used (input) during the
interaction. In the context of Comparative Usability Testing Research of DeepAI and SciteAI: A
User Interface (UI) and User Confidence (UC) Perspective efficiency refers to how well users
can achieve their goals and tasks within the systems while minimizing cognitive effort, time, and
resources.

IV. DATA COLLECTION PROCEDURE concerning EFFICIENCY parameter (Add

additional sheets as needed.)

To evaluate the efficiency parameter during usability testing of DeepAI and SciteAI in opening
and generating output, the following data collection procedure will be used:

1. Task Scenarios: Throughout the usability testing sessions, participants will be prompted
to engage in tasks utilizing both DeepAI and SciteAI platforms. These tasks may involve
actions like accessing the AI websites, scrutinizing generated outputs, and appraising
research information available within the interfaces. By performing these tasks, users will
provide valuable insights into the usability and effectiveness of both DeepAI and SciteAI
from a practical user perspective.
2. Task Completion Time: In order to assess the efficiency of user interactions, it is
essential to record the duration taken by users to fulfill each designated task. Throughout
the usability testing sessions, participants will be prompted to engage in various tasks
using both DeepAI and SciteAI platforms.
3. Error Rates: Throughout the usability testing sessions, the frequency and nature of
errors encountered during task execution will be closely monitored. Participants will
engage in various tasks using DeepAI and SciteAI interfaces, including actions such as
accessing the AI website, analyzing output data, and assessing research information. Any
errors encountered during the execution of these tasks, such as incorrect inputs, system
crashes, or misinterpretations of information, will be documented and analyzed to
identify patterns and areas for improvement in both platforms.

V. DATA COLLECTION INSTRUMENT/S or TOOL/S in relation to the

EFFICIENCY parameter of the specific usability study (Attach Questionnaire,
Observation Sheet, Checklist, Data Sheet, etc.)
 Welcome to Usability Testing Research in titled Comparative Usability Testing
Research of DeepAI and SciteAI: A User Interface (UI) and User Confidence (UC)
Perspective. This usability checklist is tailored to assess the efficiency of the DeepAI and
SciteAI. This research delves into the User Interface (UI) and User Confidence (UC) aspects of
both platforms, aiming to provide comprehensive insights into their usability. Below are the
key points included for assessment.

USER GROUP 1 USER GROUP 2

Trait no. 1 Not technologically Literate Technologically Literate

Trait no. 2 No Prior Exposure on Ai’s Accustomed on ai platforms

Trait no. 3 No prior experience with Accustomed on using deepai and sciteai

Trait no. 4 Age range(16-20) Age range(20-29)

Trait no. 5 High School Student College Student

Name:

ID no.:

Gender:

User Group:

No. Questions Yes/No

1 Would you say completing tasks using DeepAI and

SciteAi is easy?

2
Did you encounter any challenges that affected your
efficiency using these systems?
3 Are you satisfied with the overall efficiency of
DeepAi and SciteAi?

4
Were there any features of DeepAi and SciteAi that
hindered your efficiency?

(Observation sheet): Recording time

1. Time taken by the user to complete the tasks

Task 1_____min,______Errors

Task 2_____min______Errors

TASK 1: Open the DeepAi and SciteAi

No. Completing the task in minutes Success (YES/NO) Errors

9
TASK 2: Generate output in DeepAi and Sciteai and evaluate the accuracy and reliability

No. Completing the task in minutes Success (YES/NO) Errors

9
ERGONOMICS 2 (COGNITIVE ERGONOMICS)

Laboratory Activity No. 5

Name of the Activity: DEFINING USER SATISFACTION AND DESIGNING DATA
COLLECTION TOOL/S

Student’s Last Name and First Name: Saligo Rogie_______________________________

Year and Section: 3-E Date of Submission:04/24/2024

I. TITLE OF THE USABILITY STUDY

Comparative Usability Testing Research of DeepAI and SciteAI: A User Interface (UI) and User
Confidence (UC) Perspective_____________________________________________________

I. STATEMENT OF THE PROBLEM component relating to USER SATISFACTION

In the context of this comparative usability testing research on DeepAI and SciteAI, the
primary problem is to assess and compare user satisfaction with the user interfaces of these AI
platforms. Specifically, the study aims to address the following components related to user
satisfaction:

1. How do users perceive and evaluate the usability of DeepAI and SciteAI interfaces?
2. What factors influence user satisfaction with the UI design of DeepAI and SciteAI?
3. To what extent does user satisfaction impact overall acceptance and adoption of AI
technologies provided by DeepAI and SciteAI?

II. Definition of USER SATISFACTION in the Context of the Usability Study

In the context of this usability study, user satisfaction refers to the overall perception and
feeling of users towards the user interfaces (UI) of DeepAI and SciteAI. It encompasses users'
subjective assessment of usability, ease of interaction, and efficiency when utilizing these AI
platforms. Key aspects of user satisfaction include:

User Interface: The holistic experience users have while interacting with the AI platforms,
encompassing factors such as responsiveness, visual appeal, and engagement level.

Efficiency and Effectiveness: Users' perception of how efficiently and effectively they can
utilize the functionalities offered by DeepAI and SciteAI to meet their goals and expectations.

User Confidence: The level of trust and assurance users have in the accuracy, reliability, and
relevance of the AI-generated outputs presented by these platforms.
III. DATA COLLECTION PROCEDURE concerning USER SATISFACTION
parameter (Add additional sheets as needed.)

1. Pre-test Data Collection: Before the usability testing the observer will administer
pre-test surveys or interviews to gather baseline information on participants' expectations
and experiences with AI platforms and provide introductory materials about DeepAI and
SciteAI to familiarize participants.
2. Usability Testing:During usability testing, participants will be directed to perform
designated tasks on both platforms, encouraged to vocalize their thoughts throughout to
capture real-time reactions and frustrations, while interactions, task completion times,
and user feedback will be recorded.
3. Post-test Data Collection:Before the usability testing the observer will administer
post-test surveys or interviews immediately afterward to gather detailed feedback on
usability, satisfaction, and influential factors, and explore participants' opinions on the
impact of satisfaction on technology adoption.
4. User Satisfaction Ratings: Upon task completion, participants will be asked to evaluate
the efficiency of DeepAI or SciteAI using a Likert scale, rating their satisfaction levels
from 1 (very unsatisfied) to 5 (very satisfied). Their assessments will encompass aspects
such as task completion time, ease of use, user experience, and user confidence,
facilitating a comprehensive evaluation to glean nuanced feedback on the platforms'
performance and identify areas for enhancement.
5. Data Analysis: After the completion of usability testing, the collected data will be
meticulously examined to unveil prevailing trends and patterns. Thematic analysis will be
employed on interview transcripts and open-ended responses to unearth deeper insights
into participants' experiences and perceptions. This methodical approach aims to extract
rich qualitative data, enabling a comprehensive understanding of user interactions and
uncovering nuanced perspectives that may inform further improvements to the user
experience.

IV. DATA COLLECTION INSTRUMENT/S or TOOL/S in relation to the USER

SATISFACTION parameter of the specific usability study (Attach Questionnaire,
Observation Sheet, Checklist, Data Sheet, etc.)
 Welcome to Usability Testing Research in titled Comparative Usability Testing
Research of DeepAI and SciteAI: A User Interface (UI) and User Confidence (UC)
Perspective. This usability checklist is tailored to assess the efficiency of the DeepAI and
SciteAI. This research delves into the User Interface (UI) and User Confidence (UC) aspects of
both platforms, aiming to provide comprehensive insights into their usability. Below are the
key points included for assessment.

USER GROUP 1 USER GROUP 2

Trait no. 1 Not technologically Literate Technologically Literate

Trait no. 2 No Prior Exposure on Ai’s Accustomed on ai platforms

Trait no. 3 No prior experience with Accustomed on using deepai and sciteai

Trait no. 4 Age range(16-20) Age range(20-29)

Trait no. 5 High School Student College Student

Name:

ID no.:

Gender:

User Group:

Questionnaire:
To measure how satisfied users are, we'll use a questionnaire. This survey will be given out after
tasks are finished and will include these questions.

1 - Strongly Disagree
2 - Disagree
3 - Neutral
4 - Agree
5 - Strongly Agree

1 2 3 4 4

1 Are you satisfied with the overall usability of

DeepAI?

2 Are you satisfied with the overall usability of

SciteAI?

3 Did you find DeepAI easy to navigate?

4 Did you find SciteAI easy to navigate?

5 Are you satisfied with the speed of task completion

using DeepAI?

6 Are you satisfied with the speed of task completion

using SciteAI?

7 Are you confident in using DeepAI to complete

tasks?

8 Are you confident in using SciteAI to complete

tasks?

9 Are you satisfied with the user interface of DeepAI?

10 Are you satisfied with the user interface of

DeepAI?

11 Are you confident with the accuracy and reliability

of the generated output of DeepAI?

12 Are you confident with the accuracy and reliability

of the generated output ofSciteAi?