This project is the implementation template for HW 0 and HW 1 for both the programming and non-programming tracks

Overview

S22-W4111-HW-1-0:
W4111 - Intro to Databases HW0 and HW1

Introduction

This project is the implementation template for HW 0 and HW 1 for both the programming and non-programming tracks.

HW 0 - All Students

You have completed the first step, which is cloning the project template.

Note: You are Columbia students. You should be able to install SW and follow instructions.

MySQL:

  • Download the installation files for MySQL Community Server..

    • Make sure you download for the correct operating system.
    • If you are on Mac make sure you choose the correct architecture. ARM is for Apple silicon. x86 is for other Apple systems.
    • On Windows, you can download and use the MSI.
  • Follow the installation instructions for MySQL. There are official instructions and many online tutorials.

  • Remember your root user ID and password, that you set during installation. Also, choose "Legacy Authentication" when prompted.

    • If you forget your root user or password, you are on your own. The TAs and I will not fix any problems due to forgetting the information.
    • Also, if you say something like, "It did not prompt me for a user ID and password when I instaled ... ..," we will laugh. We will say something like, ""Sure. 20 million MySQL installations asked for the information, but it decide to not to ask you."
    • If you tell us that you are sure that you are entering the correct user ID and password we will laugh. We will say something like, "Which is more likely. That a DATABASE forgot something or" you did?"
  • You only need to install the server. All other SW packages are optional.

Anaconda:

  • I strongly recommend uninstalling any existing version of Anaconda. If you choose not to uninstall previous versions, you may hit issues. You are on your own if you hit issues due to conflicting versions of Anaconda during the semester.

  • Download the most recent version of Ananconda..

  • Follow the installation instructions. Choose "Install for me" when prompted. If you hit a problem and I find your Anaconda installation in the wrong directory, you are on your own. If you say something like, "But, it did not give me that option," you can guess what will happen.

DataGrip:

  • Download DataGrip. Make sure you choose the correct OS and silicon.

  • Follow the installation instructions.

  • Apply for a student license.

  • When you receive confirmation of your student license, set the license information in DataGrip.

HW0: Non-Programming

Step 1: Initial Files

  1. Create a folder in the project of the form _src, where is your UNI I created an example, which is dff9_src.

  2. Create a file in the directory _HW0.

  3. Copy the Jupyter notebook file from dff9_src/dff9_HW0.ipynb into the directory you created and replace dff9 with your UNI.

  4. Do the same for dff9_HW0.py

Step 2: Jupter Notebook

  • Start Anaconda.

  • Open Jupyter Notebook in Anaconda.

  • Navigate to the directory where you cloned the repository, and then go into the folder you created.

  • Open the notebook (the file ending in .ipynb).

  • The remaining steps in HW0: Non-Programming are in the notebook that you opened.

HW 0: Programming

  • Complete the steps for HW0: Non-Programming.

  • The programming track is not "harder" than non-programming. The initial set up is a little more work, however.

  • Download and install PyCharm. Download and install the professional edition.

  • Follow the instructions to set the license key using the JetBrains account you used to get the DataGrip licenses.

  • Start PyCharm, navigate to and open the project that you cloned from GitHub.

  • Follow the instructions for creating a new virtual Conda environment for the project.

  • Select the root folder in the project, right click and add a new Python Package named _web_src. My example is dff9_web_src.

  • Copy the files from dff9_web_src into the package you created.

  • Follow the instructions for adding a package to your virtual environment. You should add the package flask.

  • Right click on your file application.py that you copied and select run. You will see a console window open and this will show a URL. Copy on the URL.

  • Open a browser. Paste the URL and append '/health'. My URL looks like http://172.20.1.14:5000/health. Yours may be a little different.

  • Hit enter. You should see a health message. Take a screenshot of the browser window and add the file to the directory. My example is ""

Owner
Donald F. Ferguson
Senior Technical Fellow, Chief SW Architect, Ansys, Inc. Adjunct Professor, Dept. of Computer Science, Columbia University. CTO and Co-Founder, Seeka.TV
Donald F. Ferguson
Automated Exploration Data Analysis on a financial dataset

Automated EDA on financial dataset Just a simple way to get automated Exploration Data Analysis from financial dataset (OHLCV) using Streamlit and ta.

Darío López Padial 28 Nov 27, 2022
Python scripts aim to use a Random Forest machine learning algorithm to predict the water affinity of Metal-Organic Frameworks

The following Python scripts aim to use a Random Forest machine learning algorithm to predict the water affinity of Metal-Organic Frameworks (MOFs). The training set is extracted from the Cambridge S

1 Jan 09, 2022
This is an example of how to automate Ridit Analysis for a dataset with large amount of questions and many item attributes

This is an example of how to automate Ridit Analysis for a dataset with large amount of questions and many item attributes

Ishan Hegde 1 Nov 17, 2021
A CLI tool to reduce the friction between data scientists by reducing git conflicts removing notebook metadata and gracefully resolving git conflicts.

databooks is a package for reducing the friction data scientists while using Jupyter notebooks, by reducing the number of git conflicts between different notebooks and assisting in the resolution of

dataroots 86 Dec 25, 2022
MoRecon - A tool for reconstructing missing frames in motion capture data.

MoRecon - A tool for reconstructing missing frames in motion capture data.

Yuki Nishidate 38 Dec 03, 2022
Sample code for Harry's Airflow online trainng course

Sample code for Harry's Airflow online trainng course You can find the videos on youtube or bilibili. I am working on adding below things: the slide p

102 Dec 30, 2022
Projects that implement various aspects of Data Engineering.

DATAWAREHOUSE ON AWS The purpose of this project is to build a datawarehouse to accomodate data of active user activity for music streaming applicatio

2 Oct 14, 2021
Top 50 best selling books on amazon

It's a dashboard that shows the detailed information about each book in the top 50 best selling books on amazon over the last ten years

Nahla Tarek 1 Nov 18, 2021
Gathering data of likes on Tinder within the past 7 days

tinder_likes_data Gathering data of Likes Sent on Tinder within the past 7 days. Versions November 25th, 2021 - Functionality to get the name and age

Alex Carter 12 Jan 05, 2023
Show you how to integrate Zeppelin with Airflow

Introduction This repository is to show you how to integrate Zeppelin with Airflow. The philosophy behind the ingtegration is to make the transition f

Jeff Zhang 11 Dec 30, 2022
Reading streams of Twitter data, save them to Kafka, then process with Kafka Stream API and Spark Streaming

Using Streaming Twitter Data with Kafka and Spark Reading streams of Twitter data, publishing them to Kafka topic, process message using Kafka Stream

Rustam Zokirov 1 Dec 06, 2021
Spectral Analysis in Python

SPECTRUM : Spectral Analysis in Python contributions: Please join https://github.com/cokelaer/spectrum contributors: https://github.com/cokelaer/spect

Thomas Cokelaer 280 Dec 16, 2022
Bearsql allows you to query pandas dataframe with sql syntax.

Bearsql adds sql syntax on pandas dataframe. It uses duckdb to speedup the pandas processing and as the sql engine

14 Jun 22, 2022
EOD Historical Data Python Library (Unofficial)

EOD Historical Data Python Library (Unofficial) https://eodhistoricaldata.com Installation python3 -m pip install eodhistoricaldata Note Demo API key

Michael Whittle 20 Dec 22, 2022
cLoops2: full stack analysis tool for chromatin interactions

cLoops2: full stack analysis tool for chromatin interactions Introduction cLoops2 is an extension of our previous work, cLoops. From loop-calling base

YaqiangCao 25 Dec 14, 2022
A probabilistic programming library for Bayesian deep learning, generative models, based on Tensorflow

ZhuSuan is a Python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and

Tsinghua Machine Learning Group 2.2k Dec 28, 2022
Using Data Science with Machine Learning techniques (ETL pipeline and ML pipeline) to classify received messages after disasters.

Using Data Science with Machine Learning techniques (ETL pipeline and ML pipeline) to classify received messages after disasters.

1 Feb 11, 2022
PyEmits, a python package for easy manipulation in time-series data.

PyEmits, a python package for easy manipulation in time-series data. Time-series data is very common in real life. Engineering FSI industry (Financial

Thompson 5 Sep 23, 2022
pyETT: Python library for Eleven VR Table Tennis data

pyETT: Python library for Eleven VR Table Tennis data Documentation Documentation for pyETT is located at https://pyett.readthedocs.io/. Installation

Tharsis Souza 5 Nov 19, 2022
Datashredder is a simple data corruption engine written in python. You can corrupt anything text, images and video.

Datashredder is a simple data corruption engine written in python. You can corrupt anything text, images and video. You can chose the cha

2 Jul 22, 2022