BouWatch is a company that provides temporary security solutions in the form of a camera system (mounted on a mast) with off-site monitoring. This is largely used by clients that wish to protect equipment at construction sites during times no-one is present, but can also be used for various other use cases due to the versatile nature of their system.

Due to the constantly changing conditions at a construction site, it’s not unlikely that the surveillance mast is moved around (by workers). While this doesn’t have to cause problems, moving the mast in a way that creates surveillance a blind spot may have detrimental effects to the security of the site. Operators of the control room will have to ensure full coverage of the site every time a mast is moved, to ensure they still have full coverage of the desired area. Unfortunately, it happens all too often that the operators aren’t informed of the movement.

Assignment

In order to prevent cases where the operators find out too late full coverage has been lost, FIGO (the R&D department of BouWatch) instructed us to come up with an automatic solution for detecting mast movement. Our goal: use only periodically provided GPS location data (timestamp, latitude and longitude) to determine whether a mast has been moved from its original location.

Timestamp	Latitude	Longitude
2020-01-01 08:04:10	52.220584	6.886416
2020-01-01 08:09:10	52.220521	6.886431
2020-01-01 08:14:10	52.220588	6.886460
2020-01-01 08:19:10	52.220587	6.886246

GPS accuracy and drift

Anyone familiar with GPS systems can tell you that the data you receive is never accurate. Modern technologies such as RTK and DGPS aim to improve the GPS receiver’s accuracy (and do so successfully) but are very costly. Accuracy highly depends on the surroundings of the GPS receiver: tall buildings or a dense forest will interfere with the signal, whereas an open field with clear sight of the sky will give quite an accurate location. Without using technologies as RTK and DGPS, GPS is accurate to about 1.5 meters (best case scenario), but can be as inaccurate as 10+ meters.

The difference between your actual location and the location recorded by the GPS receiver is often called (GPS) drift. By simply comparing two recorded locations, it’s impossible to definitively say whether the difference is actual movement or inaccuracy.

Dataset and feature extraction

FIGO provided us with a dataset consisting of over seven million data points, collected by a little over seven hundred masts. As we were working with time series data, we were able to extract a lot of (useful) features over different timespans. After a lot of testing, we ended up using two rolling windows of one and two hours (12 and 24 entries respectively), and combining the features from both.

We extracted a total of over 150 features from the time series data, but found the model to work most effectively with between 15-20 features. After comparing the effect each feature had on the accuracy of the model, we ended up using 20 features, including mean absolute change, binned entropy and standard deviation (from both the window sizes).

Model Training and evaluation

We trained hundreds of models, each with different variables, until we eventually found the four best performing: Random Forest, SVC, KNN and GaussianNB.

At the time of our project, no information was being delayed to the operators at all. In machine learning terms, this would be equal to a model that outputs false-negatives 100% of the time. Because of the high security risk involved with false-negatives (a site without full camera coverage) and the only downside of false-positives being an operator having to check the camera feeds, we opted to prioritise sensitivity over specificity for our model.

Results

Using a validation set consisting of data never seen by the model before (during training and testing), we managed to achieve a sensitivity of 99.4%, a specificity of 76.5% and a total accuracy of 98.1%. This means that we were able to classify mast movement correctly 99.4% of the time on the first GPS location after movement, while only falsely predicting movement in 23.5% of cases.

If you have any questions about this project, or one of my other projects, please reach out using the contact form on this website, or directly via jasper@sijge.rs.