dat4sem2018fall

Getting Things Done with Python

Purpose

The purpose of this subject is a) to teach the students the programming language Python 3 and b) to apply the language, its features, and third party libraries to “get things done”. That is, the course aims to enable the students to collect, process, and visualize data using various techniques. The main goal is to allow the students to use Python as a tool in their later careers to quickly analyse problems, find answers to business questions, etc. Objectives

Knowledge

Basics

We will learn about Python’s basic data types, basic data structures, control structures, expressions, statements, operators, and program operation. Additionally, we will learn how to use “Jupyter Notebooks”, as interactive programming environment and its application for knowledge presentation. Data Collection We will learn how to automatically download files from the web, scrape text and images from web pages, and how to read and write various file formats, such as text files, CSV files, JSON files, Excel files, etc. Data Visualization We will learn how to plot data into various plot styles, such as, line plots, scatter plots, bar plots, on maps, etc. using different technologies, such as “matplotlib”, “pygal”, and “bokeh”. Data Science We will apply some common algorithms in data science, such as “KMeans”, “Mean Shift”, “Page Rank”, etc. Additionally, we will learn how to make use of the most prominent science libraries “NumPy” and “Pandas” for effective and efficient data processing. Image Processing We will have a look at basic image processing tasks, such as, reading image files, morphological operations, colour spaces, and the application of “OpenCV” to process images and streams of images automatically. Automation On top of automatic web scraping, we will have a look at UI automation and the “Selenium” framework to let computers perform the boring and repetitive tasks.

Skills

After completing this course, the students will be able to:

Competencies

Due to the project-based design of the course, the students will become competent in collecting various types of data, formulate problems about this data, implement solutions to given problem statements, and to present results on an abstract as well as technical level. Additionally, the students will gain experiences in code reviews by reviewing Python code of their fellow students.

Schedule

Intro, Getting Started, and Whirlwind Tour

Literature

Exercises

See lecture notes.

Study activity

Lists, Loops, and Conditional Statements, Functions, Modules, and Dictionaries

Literature

Exercises

See lecture notes.

Study activity

Working with Files, CLI Programs, Object-orientation, Exceptions

Literature

Exercises

Study activity

Intro to Plotting

Literature

Exercises

Study activity

Intro to NumPy and Plotting Continued

Literature

Exercises

Study activity

Intro to Pandas

Literature

Exercises

Study activity

Multiprocessing, generators and intro to Requests

Literature

Exercises

Study activity

Webscraping Basics, Regular Expressions, and Selenium

Literature

Exercises

Study activity

Graphs and PageRank

Literature

Exercises

Study activity

MeanShift

Literature

Exercises

Study activity

Perceptron Classifier (Neural Networks)

Literature

Exercises

Study activity

Image Processing with OpenCV

Literature

Exercises

Study activity

Movement Detection

Literature

Exercises

Study activity

Project

Project Workshop 1

Literature

All above

Exercises

Study activity

Project Workshop 2

Literature

All above

Exercises

Study activity

Project Workshop 3

Literature

All above

Exercises

Study activity

Course Wrap-up

Literature

All above

Exercises

Study activity

Assignments & Studypoints

You earn studypoints (SP) by solving the assignments from the list below. There are in total 9 assignments during the course. This number may chage as itdepends on the precise amount of groups to be formed in the first session.

Task Hand-out Hand-in Studypoints URL
Selection of Datasets 28. Aug. 2018 3. Sep. 12:00 10 mandatory -
Group Gifted Perception + Review 18. Feb. 2018 23. Feb. 23:55 10 https://github.com/HawkDon/Python_Assignment1
Group Foolish Supermarket + Review 25. Sep. 2018 30. Feb. 23:55 10 https://github.com/Zurina/Dataset/blob/master/README.md
Group Plain Product + Review 2. Oct. 2018 7. Oct. 23:55 10 https://github.com/MikkelHansen95/dataset
Group College Impossible + Review 8. Oct. 2018 21. Oct. 23:55 10 https://github.com/BoMarconiHenriksen/impossibleCollegeDataset
Project 1 + Review - - 10  
Project 2 + Review - - 10  
Project 3 + Review - - 10  
Project Hand-in - 19. Dec. 2018 (23:55) -  

As you can see in the table above can you earn 130 SP for all the exercises. You have to get at least 80% of the SP, which is a requirement to be eligible for the exam.

Exam

Examination form:

The project is mandatory for the exam, where the project is considered for grading (eksamensgrundlag).

Assessment

One single grade is given according to the 7-point grading scale.

Admission criteria

The student must fulfill the mandatory learning activities.