Pragma was engaged to demonstrate the potential products, such as data lakes and analytics dashboards, that could be built and utilised by an Insights and Analytics team under a Service Delivery Transformation project. This capability would work beside and empower others such as business intelligence, behavioural insights and nudge design.

The Department already collected and utilised data such as channel, visa and arrival/departure data for decision making processes and forecasting. Relationships and trends within these datasets have the ability to provide valuable insights. Even greater value was found by searching for less obvious relationships in new datasets and sources inside and outside the Department.

Pragma worked with multiple stakeholders within DHA as well as other state and federal agencies, foreign governments and the private sector to discover reliable and relevant data sources and datasets.

Pragma created a data lake using Amazon Web Services to demonstrate and test the ability to generate analytics and insights with an initial primary focus on temporary visa holders in and outside of Australia. Other analyses included machine learning algorithms applied to channel data to predict ways the public would interact with the Department in a variety of possible scenarios.

A data lake is capable of ingesting existing databases as well as live data feeds such as APIs or reporting tools. The data can be structured or unstructured which gives an advantage over traditional data warehouses. This flexibility allowed Pragma to pivot to new data sources as they became available or relevant. Given the complexity of the tools, Pragma held multiple in-person and online walk throughs and tutorials for stakeholders.