The Home Office Border Force is responsible for securing the UK border; it uses various large datasets to analyse cross-border activity and continuously seeks to improve its analysts’ efficiency.
Task
The Home Office Border Force needed to integrate various large datasets in order to reduce the time that analysts took to identify illegal immigrants. As a Capgemini Integration Engineer, Unusual Systems staff member was engaged with a small cross-industry team to prototype the extraction, transformation and loading of several large datasets into a single data warehouse, based on Palantir’s distributed database.
Activities
- Cleaned, normalised and fused the (“ETL”) datasets
- Wrote Java plugins for the analytic toolset Palantir to query the distributed (shared) dataset
- Designed models to identify and explain suitable data distribution approaches to stakeholders
- Demonstrated prototype
- Advised analysts on new techniques for the new data
- Wrote the technical case for the following (winning) proposal for an operational system
Results
The prototyping proved the warehouse would save the analysts both calendar and effort time. In some not uncommon cases wait times of several weeks were reduced to a few seconds. The Home Office accepted our proposal and funded a several-million pound project for an operational system.