Experiments
Our evaluation was designed to be rigorous and representative of real-world usage. For both the 30-minute and 60-minute time horizons, we evaluated predictions on 100 randomly selected stations, sampling their occupancy status 48 times daily (every 30 minutes) for a full week.
The model was benchmarked against a remarkably strong baseline: the “Keep Current State” approach. This baseline simply assumes that the number of available ports a certain number of minutes (H) in the future will be exactly the same as the current number.
While simple, this baseline is very hard to beat, especially over short horizons. For example, our data showed that on the US East Coast, never more than 10% of ports change their availability state within a 30-minute block. Since most of the time the state doesn’t change, the simplest prediction — no change — is correct most of the time, making the task of adding predictive value extremely difficult.
We focused on two key metrics to measure the model’s accuracy for predicting the exact number of free ports: mean squared error (MSE) and mean absolute error (MAE). A ratio of MSE/MAE ≥ 1 free port measures the accuracy of the most critical binary task for the user: “Will I find at least one free port (Yes/No)?”