Data Challenge

One of our biggest challenges was to create consistency across wildly differing data sets held within local authorities. Whilst some LA’s had developed platforms for managing and sharing this data already, some relied on bespoke web sites (API’s) whilst some only had leaflets or flyers in doctors' surgeries or public areas, to align with the policy of making their data available.

We needed to be able to import, store and maintain the data and display it to our users in a consistent and comparable way according to their search criteria/needs. We also needed to make it simple for local authorities to maintain so that they only had to enter the data once in any one platform.

Our business analysts, worked directly with pilot local authorities and undertook wider desk research across a large sub section of other local authorities, to understand the structure of the data and the various categories and subcategories used. They worked closely with the UCD team to understand user need and how users wanted to interact with the data. They also worked with the technical team to ensure that the result was a consistent data model which worked both technically and for local authorities.

A solution to the problem was to use the Open Referral framework as a vehicle for standardising the sharing of data, and Talent, engaged with the DfE into the open referral community helping shape the future vision of the framework and working together with its sponsors. This framework defines standards for sharing data over API’s and also the structure of that data, allowing anyone to consume it in a known format, something that the travel industry has been doing for years with hotel bookings, eg trivago, booking.com, with huge benefits within this sector for their users.

The result of this was to develop our own API using this Open Referral standard, with the following advantages:

1. LA’s who have no platform can enter their data in the national database AND then can pull their data into a platform of their choice, when needed.

2. LA’s who already have a platform can allow the service directory to PULL their Local data into the national database

Result:

Impact on administration is minimal and LA’s can enter data once in the platform of their choice

Technical Approach:

The Talent team approached the project with the mindset to align with the design standards and

standard agile model but bring in a disruptive mindset with new technologies. The Talent team

embedded themselves into the blended team, working within the Alpha / Beta agile framework and designed and developed a platform built around user needs.

Leveraging the Azure DfE platform, SaaS components were employed to bring Security, Stability and Scalability to the platform. Advising the client on technology was a large part of our model and we successfully brought in new CI/CD platform hosted on github, using open-source code patterns and seducing costs to the client by designing lower environment architecture to be locked down to office Wifi and specific IP address ranges. The net impact was to reduce the overall hosting costs by three quarters but keep security at the same time, with production environment and load test protected with OWASP and DDOS protection from the Application Gateway resources Azure provide.

The service supported various roles from DfE admins, organisation admins and referral users as well as being publicly accessible to members of the public. This meant engaging an authentication and authorisation platform, with Talent recommending GOV OneLogin, and this was successfully implemented as the first service on the portfolio to use this strategic authentication platform for government services.

The re-use of other platforms aligning to the technology codes of practice was at the forefront or our design, and utilisation of GOV notify was implemented to provide our channels of communication to our users.

Automation was also a key deliverable by the Talent team, with security and code analysis tools

employed to allow the team to receive push notifications when something needed our attention.

Examples of these are:

1. Sonarqube code analysis via github actions

2. NCSC (National Cyber Security Centre) webcheck to receive automated alerts on outside-in tests from this agency, eg cert issues

3. Github dependabot, pushing security alerts around 3rd party libraries we use in our code.

4. Load testing on a daily basis, bring up a complete environment, equal to that of production, then performing the test with locust.io and then bringing it down, thus saving costs and giving insights to changes to our code and infrastructure and the effect on performance. 

Talent also introduced the concept of containerisation, allowing more flexibility for where our services are deployed and reducing cloud vendor lock in, as well as a common monitoring platform via Grafana. The Grafana dashboard was introduced and is something that is now being considered across the VCF portfolio, as it allows a single portal for the following data

1. Azure metrics via Azure monitor, allowing the team to easily see issues and remediate

2. Azure costs, allowing the business to get real time information and costs forecasts on their hosting

3. GA4 user interaction analytics, allowing our business KPIs to be fully transparent and easily show and tell this information to the senior leadership teams