Where, when
and how long.

Project 1 of MAST30034 was an individual quantitative analysis of the New York Taxi & Limousine Commission trip records: pick a question, clean a very large real dataset, explore it visually and back the answer with a statistical model.

I chose 2019 and a practical question: can we tell a passenger how long a yellow-cab ride will take, from where and when it starts and what the city is doing that day?

A PySpark notebook that cleaned 84 million rows in four documented rounds, joined NOAA Central Park weather, NYC permitted events and NYPD collisions, drew Folium choropleths of every zone and fitted an elastic-net linear regression with a hand-written 10-fold cross-validation, because Spark's own CrossValidator would not run on my laptop.

This site re-runs those exact rules with DuckDB on TLC's current files. After the first step it lands within 0.003% of every row count the notebook printed.

Manhattan is the taxi system: 86% of cleaned trips start and end there, and the busiest pickup zone is Upper East Side South. Rides are slowest from 4 pm (mean 17.2 min) and quickest from 2 am (11.3 min).

The model kept 83 of 579 features. Airports dominate it: a LaGuardia pickup adds 14.3 minutes and JFK 12.0. Every weather variable was shrunk to zero; only collisions (+0.035 min each) survived. It explains about 37% of the variance, an honest result for a straight-line model.