Data Analysis and Programming 📊💻

Fundamentals of AI-Driven Algorithm Development and Data Science

Programming is about getting a computer to do what you (the user) want it to do. It’s about trying to make the future less painful. It’s about making things easier for our teammates. It’s about getting things wrong and being able to bounce back. It’s about forming good habits. It’s about understanding your toolset. The Pragmatic Programmer, 2020

Syllabus

Welcome to Data Analysis and Programming (DAP)!

In this book, you will find all the information you need to succeed in the course.

General Information

• Level: Bachelor of Science
• Faculty: Behavioural, Management and Social Sciences (BMS)
• Module: 202300167 (see on Osiris)
• Credits (ECTS): 3
• Lecturer: Breno Alves Beirigo (b.alvesbeirigo@utwente.nl)

📜Course Description

This course introduces students to

1. essential data analysis techniques and
2. computer programming fundamentals.

Essential data analysis techniques

Students will learn basic- and intermediate-level features of spreadsheet software such as MS Excel and Google Sheets, covering topics like

• tables,
• pivot tables,
• graphs,
• formula manipulation,
• worksheet layout,
• (conditional) formatting,
• proposition logic in formulas, and
• use of basic formulas for data retrieval and manipulation.

They will also develop the programming skills necessary to perform data analysis using Python libraries such as Pandas, NumPy, and Matplotlib, enabling them to automate processes and extract insights from large datasets.

Computer programming fundamentals

Students will be introduced to Python programming, focusing on real-world Industrial Engineering & Management problems. The course covers essential concepts such as

• functions,
• arguments,
• return values,
• variables,
• data types,
• conditionals,
• Boolean expressions,
• loops,
• exception handling,
• bug fixing,
• unit testing, and
• utilizing third-party libraries.

🎯Course Aims

This course prepares students for more advanced programming languages and concepts typically encountered in the MSc stage of engineering programs. The aims are as follows:

• To introduce students to data analysis using Python libraries and spreadsheet software.
• To equip students with the ability to write, test, and debug Python code effectively.
• To develop students’ understanding of fundamental programming concepts, including functions, variables, conditionals, and loops.
• To foster students’ skills in utilizing third-party libraries, handling exceptions, and fixing bugs.
• To enhance students’ file handling operations and reading/writing data proficiency.

🧠Learning Objectives

By the end of this course, students will be able to:

• Perform data analysis tasks (cleaning, manipulation, visualization) using spreadsheet software and Python libraries to extract insights and make informed decisions.
• Demonstrate a solid understanding of Python syntax, including variables, data types, and operators.
• Write and execute Python code to implement various functions, using appropriate arguments and return values.
• Utilize conditionals and Boolean expressions to control the flow of program execution.
• Employ loops effectively to perform repetitive tasks and iterate over data structures.
• Handle exceptions and debug Python code to identify and resolve errors.
• Write unit tests to verify the correctness of their code and ensure robustness.
• Access and utilize third-party libraries to leverage pre-existing functionality.
• Read and write data to files using Python’s file-handling capabilities.

⏳Study Load (ECTS)

The DAP course has a total study load of 3 ECTS¹ (84 hours). Table 1 shows the estimated study load per activity.

The activities are as follows:

• Lecture: New concepts are introduced; lectures can be in-person or online.
• Tutorial: Aimed at practicing concepts introduced in lectures and solving assignments.
• Q&A Session: A session for asking questions about the course content right before the exam.
• Exam Review: A session to review exam results and discuss solutions. You will have the chance to inspect your exam and compare your answers with the correct ones.
• Self-study: Additional study time may be required to understand and complete the assignments.
• Exam Preparation: Time allocated for studying the course content and preparing for exams.
• Exam: An exam is held in a computer lab. You will submit your answers using a Chromebook provided by the university.

Table 1: DAP course study load.

Activity	Count	Duration (h)	Workload (h)
Lectures	8	1.75	14.00
Tutorials	10	1.75	17.50
Q&A Sessions	3	1.75	5.25
Exam Reviews	3	1.75	5.25
Self-study	10	2.00	20.00
Exam Preparation	3	5.00	15.00
Exams	3	2.00	6.00
		Total	83

Resit Exam

Note that the workload breakdown of the activities involving the resit exam is not included in Table 1. These are as follows:

• Q&A Session: 1 session × 1.75 hours = 1.75 hours
• Resit Exam: 1 exam × 2 hours = 2.00 hours
• Resit Exam Review: 1 session × 1.75 hours = 1.75 hours

📅 Schedule

The regular course schedule encompasses 10 weeks. Each week, there are two sessions: a lecture and a tutorial. There are three exams (weeks 3, 7, and 10) and a resit exam (a month after the last exam).

During exam weeks, there is an additional session for Q&A. Also, the lecture immediately after an exam is dedicated to the exam review and inspection.

The detailed schedule is as follows:

• w46 (1/10)

  • Mon, 11/11 - 10:45-12:30 | RA 4231 | Lecture
  • Mon, 11/11 - 13:45-15:30 | RA 3237 | Tutorial

• w47 (2/10)

  • Mon, 18/11 - 10:45-12:30 | RA 4334 | Lecture
  • Mon, 18/11 - 13:45-15:30 | RA 4336 | Tutorial

• w48 (3/10)

  • Mon, 25/11 - 10:45-12:30 | RA 4231 | Lecture
  • Mon, 25/11 - 13:45-15:30 | VR 583 | Tutorial
  • Wed, 27/11 - 10:45-12:30 | VR 583 | Q&A
  • Fri, 29/11 - 13:45-15:45 | NH 207 | 📝Exam 1 (Chromebook)

• w49 (4/10)

  • Mon, 02/12 - 10:45-12:30 | RA 3237 | Exam Review
  • Mon, 02/12 - 13:45-15:30 | CR 3E | Tutorial

• w50 (5/10)

  • Mon, 09/12 - 10:45-12:30 | CR 3F | Lecture (Online)
  • Mon, 09/12 - 13:45-15:30 | VR 583 | Tutorial

• w51 (6/10)

  • Mon, 16/12 - 10:45-12:30 | CR 3F | Lecture (Online)
  • Mon, 16/12 - 13:45-15:30 | VR 583 | Tutorial

• w2 (7/10)

  • Mon, 06/01 - 10:45-12:30 | RA 2231 | Lecture
  • Mon, 06/01 - 13:45-15:30 | OH 116 | Tutorial
  • Wed, 08/01 - 10:45-12:30 | RA 2231 | Q&A
  • Fri, 10/01 - 13:45-15:45 | NH 207 | 📝Exam 2 (Chromebook)

• w3 (8/10)

  • Mon, 15/01 - 10:45-12:30 | RA 3237 | Exam Review
  • Wed, 15/01 - 13:45-15:30 | RA 4334 | Tutorial

• w4 (9/10)

  • Mon, 20/01 - 10:45-12:30 | CR 3F | Lecture
  • Mon, 20/01 - 13:45-15:30 | OH 114 | Tutorial

• w5 (10/10)

  • Mon, 27/01 - 10:45-12:30 | OH 116 | Lecture
  • Mon, 27/01 - 13:45-15:30 | OH 116 | Tutorial
  • Wed, 29/01 - 10:45-12:30 | VR 583 | Q&A
  • Fri, 31/01 - 13:45-15:45 | NH 209 | 📝Exam 3 (Chromebook)

• w6

  • Fri, 07/02 - 10:45-12:30 | ------ | Exam Review

• w9

  • Mon, 24/02 - 10:45-12:30 | ------ | Q&A
  • Wed, 26/02 - 13:45-15:45 | ------ | 📝Resit (Chromebook)
  • ---, --/-- - __:__-__:__ | ------ | Exam Review

Checking Resit Details on TimeEdit

Notice that, schedule 1B on TimeEdit covers activities until week 5. From week 6 onwards, the activities are scheduled for 2A. Make sure to check the details for the resit exam on TimeEdit.

✍️ Exams and Grading

This course will be assessed through three 2-hour exams, totaling 100 points. Each exam covers the content taught up to that point in the course, with the final exam encompassing the entire syllabus. The distribution of points is as follows:

• Exam 1 (\(E1\)): 20 points
• Exam 2 (\(E2\)): 30 points
• Exam 3 (\(E3\)): 50 points

You will need to score a minimum of 60 points to pass the course.

Grade Calculation Formula

At UT, the grading system is based on a scale from 1 to 10, with 5.5 being the passing threshold. Considering the three exams, the total score (\(p = E1 + E2 + E3\)) achieved by a student will be mapped to a final grade (\(G\)) on this scale. Let \(c = 60\) be the cutting score and \(t = 100\) the total points available. The grade calculation formula is as follows.

If \(p \leq c\) (i.e., the student scored below or equal to the cutting score), the grade is calculated as:

\[ \begin{aligned} G \quad = \quad & 1 + \left( \frac{p}{c} \right) \times 4.5. & \\ \end{aligned} \]

If \(p > c\) (i.e., the student scored above the cutting score), the grade is calculated as:

\[ \begin{aligned} G \quad = \quad & 5.5 + \left( \frac{p - c}{t - c} \right) \times 4.5. & \\ \end{aligned} \]

Therefore:

\[ G = \begin{cases} 1 + \left( \dfrac{p}{60} \right) \times 4.5 & \text{if } p \leq 60 \\ 5.5 + \left( \dfrac{p - 60}{40} \right) \times 4.5 & \text{if } p > 60 \end{cases} \tag{1}\]

In Figure 1, you can see the mapping of total points achieved to the final grade.

Figure 1: Grade mapping based on total points achieved. The 5.5 grade corresponds to the passing threshold of 60 points.

Resit Exam

If a student fails to pass the course, they will have the opportunity to take a resit exam. The resit exam will cover the entire course content and will be graded out of 100 points. The content and format of the resit exam will be similar to the final exam (\(E3\)). The final grade will be calculated based on the resit exam score using Equation 1.

📋Assessment Format and Style

The assessment consists of two main components:

1. Code Reading and Critique: As AI tools have simplified the process of writing code, this course will assess your ability to read, understand, validate, edit, and test code generated by these tools to ensure it meets the requirements of a given problem. For example, you may be asked to identify errors in code snippets, explain the purpose of specific code segments, or predict the output of a given code snippet.
2. Code Writing: You will be asked to write code to solve specific problems. Additionally, you will be expected to integrate pre-existing code (e.g., generated by AI tools, from other libraries, or third-party sources) into your solutions. For example, assume you are given a code snippet produced by an AI tool that solves a specific problem. You will be asked to integrate this code into a larger program that addresses a more complex problem.

The exam format will be a digital Chromebook closed-book exam. Multiple-choice questions (MCQs) will be used to assess Code Reading and Critique, while programming assignments (e.g., Python functions to complete specific tasks) will be used to assess Code Writing.

Autograding Code

To ensure fairness and consistency, the exam will be autograded. This means that your code will be automatically evaluated based on predefined criteria. Therefore, it is essential to follow the instructions carefully and ensure that your code meets the requirements specified in the exam questions.

The automatic grading system will evaluate your code based on the correctness of the output: if your code produces the expected output, you will receive full credit; otherwise, you will receive partial credit based on the number of correct outputs.

For example, suppose you are asked to write an algorithm to find the maximum value in a list of numbers. Let’s assume five test cases are provided, each with a different list of numbers L. The expected output for each test case is as follows:

Table 2: Expected output for the maximum value in a list of numbers

Test Case	List L	Expected Output
1	[1, 2, 3, 4, 5]	5
2	[5, 4, 3, 2, 1]	5
3	[-1, -2, -3, -4, -5]	-1
4	[1]	1
5	[]	None

If your code produces the correct output for the first three test cases but fails for the last two, you will receive partial credit based on the number of correct outputs. In this case, you will receive 60% (i.e., 3/5) of the total points available for this question.

📚Materials

💻Python programming

• Al Sweigart. Automate the Boring Stuff with Python: Practical Programming for Total Beginners. San Francisco: No Starch Press, 2019 (2nd Edition)²
• Python 3.12.0 documentation (python.org)
• Learn Python - Free Interactive Python Tutorial

📊Data analysis

• Excel help & learning (microsoft.com)
• Google Sheets training and help - Google Workspace Learning Center
• Pandas 2.1.2 documentation (pydata.org)
• From data to Viz - Find the graphic you need (data-to-viz.com)

🤖 Policy on Generative AI

In this course, we encourage the use of Generative AI, such as ChatGPT, to assist with your coding. However, we expect you to use it as a tool to help you understand the concepts, not as a replacement for your learning process. AI gives you the power to quickly test new ideas, but it is your responsibility to understand the concepts behind them and apply them correctly in the right context.

You may treat AI as a “virtual TA” that can help you with coding and debugging³.

For debugging, you can use AI to help you understand error messages and suggest possible solutions. However, you should always strive to understand the error and the solution yourself.

Important

The focus on understanding is crucial, as the exams will test your comprehension of the concepts and your ability to generate code.

Prompt Examples for an AI Tutor

The following are examples of prompts you can use with an AI tutor:

• “What does this error message mean?”
• “How can I fix this error?”
• “What does this code do?”
• “How can I improve this code?”
• “What is the best way to implement this feature?”
• “What is the output of this code?”
• “How can I optimize this code?”
• “What is the best way to structure this code?”
• “What is the best way to debug this code?”
• “What is the best way to test this code?”
• “What is the best way to document this code?”
• “What is the best way to refactor this code?”

These prompts can help you use AI as a tool to enhance your learning experience. It has been shown that if you ask AI to explain the code “step-by-step,” it can help you understand the code better.

Getting Started with AI

If you are new to using AI for coding, here are some tips to get started:

1. Start Small: Begin with simple prompts and gradually increase the complexity.
2. Ask Questions: Use AI to answer specific questions you have about the code.
3. Understand the Output: Make sure you understand the AI-generated code and the reasoning behind it.

AI Tools

Here are some AI tools you can use for coding:

• GitHub Copilot (Free for verified students!)
• ChatGPT (Free limited edition!)

Guidelines for Study Periods

• Permitted Use: Students are allowed to use generative AI tools for learning, understanding concepts, and coding practice.
• Understanding Over Copying: Emphasis should be on understanding the AI-generated solutions, not just copying them. Students should be prepared to explain how and why a particular solution works.

Guidelines for Assessment Periods

• Prohibited Use: The use of generative AI tools is strictly prohibited during assessments, tests, and exams.
• Penalty for Misuse: Any use of AI tools during assessments will be considered academic misconduct and subject to penalties as per the institution’s academic integrity policy.

---

1. ECTS: European Credit Transfer System. One ECTS is equal to 28 hours of study.

2. The book uses the Mu Editor, which is a simple Python editor for beginner programmers. In this course, we use the Visual Studio Code.

3. Debugging refers to the process of identifying and fixing errors in code. It is an essential skill for programmers to develop.